Skip to content

replacer

replacer

Tabular PII replacement interface.

Classes:

Name Description
TabularPiiReplacer

Replace PII in a dataframe through one plan-driven interface.

TabularPiiReplacer(config, *, data_config, time_series=None)

Replace PII in a dataframe through one plan-driven interface.

The replacement module owns plan resolution, DAG execution, free-text span detection and rewriting, positional row identity, scope keys, synthetic value generation, and statistics. replace returns a new dataframe and never mutates the caller's frame or writes artifacts. The pipeline remains responsible for deciding whether and where to persist the resolved configuration.

Parameters:

Name Type Description Default
config ReplacePiiConfig

PII replacement configuration, including the replacement plan.

required
data_config DataParameters

Input data configuration used to validate and execute the resolved plan.

required
time_series TimeSeriesParameters | None

Optional time-series configuration used to protect ordering and grouping columns.

None

Methods:

Name Description
replace

Return a replacement result for df without mutating df.

detect

Return the column statistics replace would report for df, without replacing anything.

Source code in src/nemo_safe_synthesizer/pii_replacer/replacer.py
def __init__(
    self,
    config: ReplacePiiConfig,
    *,
    data_config: DataParameters,
    time_series: TimeSeriesParameters | None = None,
) -> None:
    self._config = config
    self._data_config = data_config
    self._time_series = time_series

replace(df, *, capture_replacement_map=False, check_pattern_coverage=True)

Return a replacement result for df without mutating df.

Parameters:

Name Type Description Default
df DataFrame

Dataframe whose planned PII values should be replaced.

required
capture_replacement_map bool

Include sensitive per-occurrence replacement provenance and accepted free-text span traces in the result. The default leaves this data unavailable so ordinary calls do not accidentally persist original PII.

False
check_pattern_coverage bool

Require each pattern to cover at least 99% of df's values. Pass False when df is a subset of data the plan was already resolved against, such as the pipeline's training split, so a subset that falls just below the threshold does not reject a plan the full input accepted.

True

Raises:

Type Description
ParameterError

If the configured plan is invalid for df.

GenerationError

If a replacement cannot be generated.

Source code in src/nemo_safe_synthesizer/pii_replacer/replacer.py
def replace(
    self,
    df: pd.DataFrame,
    *,
    capture_replacement_map: bool = False,
    check_pattern_coverage: bool = True,
) -> TransformResult:
    """Return a replacement result for ``df`` without mutating ``df``.

    Args:
        df: Dataframe whose planned PII values should be replaced.
        capture_replacement_map: Include sensitive per-occurrence replacement
            provenance and accepted free-text span traces in the result. The
            default leaves this data unavailable so ordinary calls do not
            accidentally persist original PII.
        check_pattern_coverage: Require each pattern to cover at least 99%
            of ``df``'s values. Pass ``False`` when ``df`` is a subset of
            data the plan was already resolved against, such as the
            pipeline's training split, so a subset that falls just below
            the threshold does not reject a plan the full input accepted.

    Raises:
        ParameterError: If the configured plan is invalid for ``df``.
        GenerationError: If a replacement cannot be generated.
    """
    started = time.perf_counter()
    resolved_config = resolve_replacement_config(
        df,
        self._config,
        self._data_config,
        self._time_series,
        check_pattern_coverage=check_pattern_coverage,
    )
    if resolved_config.inline_plan is not None and not resolved_config.inline_plan.columns_to_replace:
        logger.user.warning("The PII replacement plan lists no columns, so no values are replaced")
    executor = self._executor(resolved_config, capture_replacement_map=capture_replacement_map)
    execution = executor.execute(df)
    return TransformResult(
        transformed_df=execution.dataframe,
        column_statistics=execution.column_statistics,
        resolved_config=resolved_config,
        generation_statistics=execution.generation_statistics,
        replacement_time_seconds=time.perf_counter() - started,
        replacement_map=execution.replacement_map,
    )

detect(df, *, check_pattern_coverage=True)

Return the column statistics replace would report for df, without replacing anything.

The plan is resolved and PII is detected exactly as in replace, but no replacement value is generated and df is not copied or changed.

Parameters:

Name Type Description Default
df DataFrame

Dataframe whose PII should be detected.

required
check_pattern_coverage bool

Same as for replace.

True

Raises:

Type Description
ParameterError

If the configured plan is invalid for df.

GenerationError

If free-text detection fails.

Source code in src/nemo_safe_synthesizer/pii_replacer/replacer.py
def detect(self, df: pd.DataFrame, *, check_pattern_coverage: bool = True) -> DetectionResult:
    """Return the column statistics ``replace`` would report for ``df``, without replacing anything.

    The plan is resolved and PII is detected exactly as in ``replace``, but no replacement value is
    generated and ``df`` is not copied or changed.

    Args:
        df: Dataframe whose PII should be detected.
        check_pattern_coverage: Same as for ``replace``.

    Raises:
        ParameterError: If the configured plan is invalid for ``df``.
        GenerationError: If free-text detection fails.
    """
    started = time.perf_counter()
    resolved_config = resolve_replacement_config(
        df,
        self._config,
        self._data_config,
        self._time_series,
        check_pattern_coverage=check_pattern_coverage,
    )
    column_statistics = self._executor(resolved_config).detect(df)
    return DetectionResult(
        column_statistics=column_statistics,
        resolved_config=resolved_config,
        detection_time_seconds=time.perf_counter() - started,
    )