Skip to content

addresses

addresses

Split structured street-address values into a replaceable street and the locality text after it.

Classes:

Name Description
AddressParts

One street-address value split at the end of its detected street.

Functions:

Name Description
detect_address_parts

Find the street inside each distinct street-address value, keyed by column name and value text.

AddressParts(street, locality) dataclass

One street-address value split at the end of its detected street.

Example

"489 Park Blvd, Mesa, IA 76237" splits into street "489 Park Blvd" and locality ", Mesa, IA 76237".

Methods:

Name Description
replacement_street

Return the generated street inside replacement, or None when it does not end with the locality.

Attributes:

Name Type Description
street str

Leading street text that is replaced. Excluded from repr because it is PII.

locality str

Text after the street, such as city, postcode, and country, kept as written. Excluded from repr.

street = field(repr=False) class-attribute instance-attribute

Leading street text that is replaced. Excluded from repr because it is PII.

locality = field(repr=False) class-attribute instance-attribute

Text after the street, such as city, postcode, and country, kept as written. Excluded from repr.

replacement_street(replacement)

Return the generated street inside replacement, or None when it does not end with the locality.

Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/addresses.py
def replacement_street(self, replacement: str) -> str | None:
    """Return the generated street inside ``replacement``, or ``None`` when it does not end with the locality."""
    if not replacement.endswith(self.locality):
        return None
    return replacement[: len(replacement) - len(self.locality)] or None

detect_address_parts(detector, dataframe, specs)

Find the street inside each distinct street-address value, keyed by column name and value text.

GLiNER2 street spans exclude the city, postcode, and country, so a value whose only detected span is a street at its start keeps the text after that street. Values without such a span are left out and replaced whole, as is every value when no detector is configured or detection cannot run. Detection reads the original values before any column is replaced; value text is the same string form the structured generator receives.

Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/addresses.py
def detect_address_parts(
    detector: FreeTextDetector | None,
    dataframe: pd.DataFrame,
    specs: Sequence[PiiColumnPlan],
) -> dict[str, dict[str, AddressParts]]:
    """Find the street inside each distinct street-address value, keyed by column name and value text.

    GLiNER2 street spans exclude the city, postcode, and country, so a value whose only detected span is a street
    at its start keeps the text after that street. Values without such a span are left out and replaced whole, as
    is every value when no detector is configured or detection cannot run. Detection reads the original values
    before any column is replaced; value text is the same string form the structured generator receives.
    """
    if detector is None or not specs:
        return {}
    cells = list(_address_cells(dataframe, specs))
    if not cells:
        return {}
    try:
        spans = detector.detect(cells).spans
    except GenerationError as exc:
        logger.user.warning(
            "Street-address values were replaced whole because free-text PII detection could not run",
            extra={"column_count": len(specs), "reason": str(exc)},
        )
        return {}
    requested = {cell.cell_id: cell for cell in cells}
    spans_by_cell: dict[DetectionCellId, list[DetectedSpan]] = defaultdict(list)
    for span in spans:
        cell = requested.get(span.cell_id)
        if cell is None:
            raise InternalError("free-text PII detector returned a span for an unrequested street-address cell")
        detected_text(cell, span)
        spans_by_cell[span.cell_id].append(span)

    parts: dict[str, dict[str, AddressParts]] = {}
    for cell in cells:
        split = _split_at_street(cell.text, spans_by_cell.get(cell.cell_id, ()))
        if split is not None:
            parts.setdefault(cell.cell_id.column_name, {})[cell.text] = split
    return parts