free_text
free_text
¶
Detection, scoped mapping reuse, and one-pass free-text replacement.
Classes:
| Name | Description |
|---|---|
FreeTextColumnStatistics |
Mutable aggregate statistics for one planned free-text column. |
FreeTextCellDetections |
Validated detector output for the original free-text cells, grouped by cell. |
FreeTextExecutionResult |
Aggregate outputs owned by one free-text execution pass. |
FreeTextReplacementExecutor |
Replace accepted spans while preserving scoped structured mappings. |
FreeTextColumnStatistics(counts=dict(), values=dict(), sources=set(), kept_name_part_count=0)
dataclass
¶
Mutable aggregate statistics for one planned free-text column.
Attributes:
| Name | Type | Description |
|---|---|---|
counts |
dict[str, int]
|
Accepted span count per entity type. |
values |
dict[str, set[str]]
|
Distinct accepted span texts per entity type. |
sources |
set[str]
|
Detector sources that produced accepted spans. |
kept_name_part_count |
int
|
Full-name replacements that kept an original first or last name because no other name was found. |
counts = field(default_factory=dict)
class-attribute
instance-attribute
¶
Accepted span count per entity type.
values = field(default_factory=dict)
class-attribute
instance-attribute
¶
Distinct accepted span texts per entity type.
sources = field(default_factory=set)
class-attribute
instance-attribute
¶
Detector sources that produced accepted spans.
kept_name_part_count = 0
class-attribute
instance-attribute
¶
Full-name replacements that kept an original first or last name because no other name was found.
FreeTextCellDetections(spans=dict(), protected=dict())
dataclass
¶
Validated detector output for the original free-text cells, grouped by cell.
Attributes:
| Name | Type | Description |
|---|---|---|
spans |
dict[DetectionCellId, tuple[DetectedSpan, ...]]
|
Accepted, non-overlapping spans to replace. |
protected |
dict[DetectionCellId, tuple[ProtectedSpan, ...]]
|
Spans that known-value matches must not overlap. |
FreeTextExecutionResult(column_statistics, generated_replacement_count, generator_time_seconds, replacement_records)
dataclass
¶
Aggregate outputs owned by one free-text execution pass.
Attributes:
| Name | Type | Description |
|---|---|---|
column_statistics |
dict[str, FreeTextColumnStatistics]
|
Detection statistics keyed by free-text column name. |
generated_replacement_count |
int
|
Replacements generated fresh rather than reused from a mapping. |
generator_time_seconds |
float
|
Time spent inside the generator for free-text spans. |
replacement_records |
tuple[FreeTextReplacementRecord, ...]
|
Per-span records when capture was requested, otherwise empty. |
column_statistics
instance-attribute
¶
Detection statistics keyed by free-text column name.
generated_replacement_count
instance-attribute
¶
Replacements generated fresh rather than reused from a mapping.
generator_time_seconds
instance-attribute
¶
Time spent inside the generator for free-text spans.
replacement_records
instance-attribute
¶
Per-span records when capture was requested, otherwise empty.
FreeTextReplacementExecutor(generator, detector, *, base_seed, capture_replacement_map=False, find_and_replace=False, locale='en_US')
¶
Replace accepted spans while preserving scoped structured mappings.
Detection always reads complete original cells before structured columns
mutate. Replacement later uses those original offsets to build each cell
exactly once. Structured mappings are registered through the same module so
independently detected exact values and validated name components remain
consistent within record or group scope. With find_and_replace, every
cell is also searched for those structured and detected values, so
occurrences the detector missed get the same replacement.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
generator
|
ReplacementGenerator
|
Produces a replacement for each value that has no mapping yet. |
required |
detector
|
FreeTextDetector | None
|
Finds candidate spans in free-text cells, or |
required |
base_seed
|
int
|
Seed from which every replacement's deterministic seed is derived. |
required |
capture_replacement_map
|
bool
|
Whether to keep a per-span record of each original and replacement value in the execution result. |
False
|
find_and_replace
|
bool
|
Whether to also replace occurrences of known structured and detected values that the detector missed. |
False
|
locale
|
str
|
|
'en_US'
|
Methods:
| Name | Description |
|---|---|
reset |
Discard all mappings and aggregates from an earlier dataframe execution. |
detect |
Detect and validate candidate and protected spans against complete original cells. |
execute |
Apply accepted original-cell spans to every planned free-text target. |
detect_statistics |
Record the spans |
register_structured_mapping |
Expose exact values, pattern-validated name parts, and street parts to accepted spans. |
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/free_text.py
reset()
¶
Discard all mappings and aggregates from an earlier dataframe execution.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/free_text.py
detect(dataframe, specs)
¶
Detect and validate candidate and protected spans against complete original cells.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/free_text.py
execute(working, specs, group_identities, detections)
¶
Apply accepted original-cell spans to every planned free-text target.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/free_text.py
detect_statistics(dataframe, specs, group_identities, detections)
¶
Record the spans execute would replace in each free-text target, without replacing them.
Structured mappings must be registered first, as execute expects, so known-value matches are the same.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/free_text.py
register_structured_mapping(key, spec, original_text, replacement, address_parts=None)
¶
Expose exact values, pattern-validated name parts, and street parts to accepted spans.
original_text is the source cell's text form, the same value the
structured generator received, so a typed original such as an integer
API key matches the digits a detector finds in prose. key carries
its typed identity. address_parts is the detected street of a
street-address value whose locality was kept.