Skip to content

resolver

resolver

Resolve one validated PII replacement plan from configuration and data.

Classes:

Name Description
ColumnProfile

Bounded descriptive evidence about one column for plan discovery.

PlanDiscoveryInput

Data and deterministic preparation supplied to discovery adapters.

PlanDiscoverer

Internal seam for producing the heuristic baseline plan.

PlanEnhancer

Internal seam for revising a heuristic baseline plan.

HeuristicPlanDiscoverer

Initial no-op heuristic adapter, ready for later rule discovery.

Functions:

Name Description
resolve_plan

Resolve and validate one replacement plan.

resolve_replacement_config

Resolve the replacement plan and its data-to-sampler value mapping for df.

ColumnProfile(column_name, dtype, non_null_count, unique_count, unique_ratio, samples) dataclass

Bounded descriptive evidence about one column for plan discovery.

PlanDiscoveryInput(dataframe, group_column, protected_columns, column_profiles) dataclass

Data and deterministic preparation supplied to discovery adapters.

group_column is the runtime consistency signal: a configured column means replacements remain consistent within each group; None means each record is replaced independently. This derived choice is intentionally not part of the reusable replacement plan.

PlanDiscoverer

Bases: ABC

Internal seam for producing the heuristic baseline plan.

Methods:

Name Description
discover

Return a context-free valid baseline plan.

discover(discovery_input) abstractmethod

Return a context-free valid baseline plan.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
@abstractmethod
def discover(self, discovery_input: PlanDiscoveryInput) -> PiiReplacementPlan:
    """Return a context-free valid baseline plan."""

PlanEnhancer

Bases: ABC

Internal seam for revising a heuristic baseline plan.

Methods:

Name Description
enhance

Return a context-free valid replacement for baseline.

enhance(discovery_input, baseline) abstractmethod

Return a context-free valid replacement for baseline.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
@abstractmethod
def enhance(
    self,
    discovery_input: PlanDiscoveryInput,
    baseline: PiiReplacementPlan,
) -> PiiReplacementPlan:
    """Return a context-free valid replacement for ``baseline``."""

DependencyMappingDiscoverer

Bases: Protocol

Discover sampler-specific mappings for dependency-column values.

Methods:

Name Description
discover_data_to_sampler_value_mapping

Return sparse mappings for non-identity dependency values.

discover_data_to_sampler_value_mapping(dataframe, plan, catalog)

Return sparse mappings for non-identity dependency values.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
def discover_data_to_sampler_value_mapping(
    self,
    dataframe: pd.DataFrame,
    plan: PiiReplacementPlan,
    catalog: Mapping[EntityType, tuple[str, ...]],
) -> DataToSamplerValueMapping:
    """Return sparse mappings for non-identity dependency values."""

HeuristicPlanDiscoverer

Bases: PlanDiscoverer

Initial no-op heuristic adapter, ready for later rule discovery.

Methods:

Name Description
discover

Return an empty plan for later heuristic discovery.

discover(discovery_input)

Return an empty plan for later heuristic discovery.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
def discover(self, discovery_input: PlanDiscoveryInput) -> PiiReplacementPlan:
    """Return an empty plan for later heuristic discovery."""
    return PiiReplacementPlan()

resolve_plan(df, config, data_config, time_series=None, *, discoverer=None, enhancer=None, check_pattern_coverage=True)

Resolve and validate one replacement plan.

An inline plan is authoritative and bypasses discovery; a pattern of its that covers less than 99% of its column logs a warning instead of failing. Auto-discovery runs the heuristic adapter first, then the LLM enhancer configured by config.llm. When no enhancer is supplied, the default enhancer runs against a managed local vLLM server unless NSS_INFERENCE_ENDPOINT selects another service; that server stops as soon as enhancement ends. Dataframe-aware validation occurs once, after the final plan has been selected.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
def resolve_plan(
    df: pd.DataFrame,
    config: ReplacePiiConfig,
    data_config: DataParameters,
    time_series: TimeSeriesParameters | None = None,
    *,
    discoverer: PlanDiscoverer | None = None,
    enhancer: PlanEnhancer | None = None,
    check_pattern_coverage: bool = True,
) -> PiiReplacementPlan:
    """Resolve and validate one replacement plan.

    An inline plan is authoritative and bypasses discovery; a pattern of its that covers less than 99% of
    its column logs a warning instead of failing.
    Auto-discovery runs the heuristic adapter first, then the LLM enhancer
    configured by ``config.llm``. When no ``enhancer`` is
    supplied, the default enhancer runs against a managed local vLLM server
    unless ``NSS_INFERENCE_ENDPOINT`` selects another service; that server
    stops as soon as enhancement ends. Dataframe-aware validation occurs once, after the final plan has
    been selected.
    """
    if (inline_plan := config.inline_plan) is not None:
        plan = inline_plan
    else:
        discovery_input = _prepare_discovery_input(df, data_config, time_series)
        baseline = (discoverer or HeuristicPlanDiscoverer()).discover(discovery_input)
        if enhancer is not None:
            plan = enhancer.enhance(discovery_input, baseline)
        else:
            plan = _enhance_with_default_llm(config.llm, discovery_input, baseline)
        plan = _without_row_counters(df, plan, discovery_input.group_column)

    validate_plan(
        df,
        plan,
        data_config=data_config,
        time_series=time_series,
        check_pattern_coverage=check_pattern_coverage,
        warn_on_low_pattern_coverage=not config.is_auto_discovery,
    )
    return plan

resolve_replacement_config(df, config, data_config, time_series=None, *, discoverer=None, enhancer=None, mapping_discoverer=None, sampler_value_catalog=None, check_pattern_coverage=True)

Resolve the replacement plan and its data-to-sampler value mapping for df.

The plan comes from config or, for auto_discovery, from the discoverer and the LLM enhancer. When config.sampler is not set and the Nemotron-Personas data for config.locale is missing, the sampler switches to Faker with a warning, unless the plan maps ethnicity values, which only Nemotron-Personas has. The value mapping then combines built-in entries for common gender spellings, the plan's own entries, and, for auto_discovery, discovered entries for any remaining conditioner values without a sampler match, and is checked against the sampler.

A plan the user supplied is not discovered, so two mismatches that leave every column replaceable are warnings instead of errors: a pattern covering less than 99% of its column, and a non-categorical dependency column, which is ignored. Value mapping problems fail for every plan: mapping entries the sampler cannot use, and dataset values without a sampler match or entry.

Parameters:

Name Type Description Default
df DataFrame

Data the plan is validated against and whose conditioner values are mapped.

required
config ReplacePiiConfig

PII replacement configuration to resolve.

required
data_config DataParameters

Data parameters, used for grouping and protected columns.

required
time_series TimeSeriesParameters | None

Time-series parameters, used for protected columns.

None
discoverer PlanDiscoverer | None

Plan discoverer for auto_discovery; defaults to the heuristic discoverer.

None
enhancer PlanEnhancer | None

Plan enhancer; defaults to the LLM enhancer configured by config.llm.

None
mapping_discoverer DependencyMappingDiscoverer | None

Value-mapping discoverer; defaults to the LLM enhancer configured by config.llm.

None
sampler_value_catalog Mapping[EntityType, tuple[str, ...]] | None

Sampler values per entity type; loaded from the configured sampler when omitted.

None
check_pattern_coverage bool

Require each pattern to cover at least 99% of df's values; see validate_plan.

True

Returns:

Type Description
ReplacePiiConfig

A copy of config whose replacement_plan is the resolved inline plan,

ReplacePiiConfig

including its complete data_to_sampler_value_mapping, whose

ReplacePiiConfig

sampler is the one used, and whose seed is set, drawn at random

ReplacePiiConfig

when config left it unset.

When both plan enhancement and mapping discovery use the default LLM, they share one inference session, so a managed local vLLM server starts once and stops before this returns. Plan validation runs inside that session, between the two LLM steps.

Raises:

Type Description
ParameterError

If the plan is invalid; for a discovered plan, if the mapping does not match the sampler, a conditioner value has no sampler match or mapping entry and cannot be discovered, or the plan maps ethnicity values while Nemotron-Personas data is missing; or if the sampler is set to Nemotron-Personas and its dataset is missing.

Example

With a plan where first_name depends on sex, values "F" and "Woman" resolve to {"sex": {"F": ["female"], "Woman": ["female"]}} without an LLM, while a value such as "X" needs a mapping entry.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
def resolve_replacement_config(
    df: pd.DataFrame,
    config: ReplacePiiConfig,
    data_config: DataParameters,
    time_series: TimeSeriesParameters | None = None,
    *,
    discoverer: PlanDiscoverer | None = None,
    enhancer: PlanEnhancer | None = None,
    mapping_discoverer: DependencyMappingDiscoverer | None = None,
    sampler_value_catalog: Mapping[EntityType, tuple[str, ...]] | None = None,
    check_pattern_coverage: bool = True,
) -> ReplacePiiConfig:
    """Resolve the replacement plan and its data-to-sampler value mapping for ``df``.

    The plan comes from ``config`` or, for ``auto_discovery``, from the
    discoverer and the LLM enhancer. When ``config.sampler`` is not set and
    the Nemotron-Personas data for ``config.locale`` is missing, the sampler
    switches to Faker with a warning, unless the plan maps ethnicity
    values, which only Nemotron-Personas has. The value
    mapping then combines built-in entries for common gender spellings, the
    plan's own entries, and, for ``auto_discovery``, discovered entries for any
    remaining conditioner values without a sampler match, and is checked
    against the sampler.

    A plan the user supplied is not discovered, so two mismatches that leave
    every column replaceable are warnings instead of errors: a pattern covering
    less than 99% of its column, and a non-categorical dependency column, which
    is ignored. Value mapping problems fail for every plan: mapping entries the
    sampler cannot use, and dataset values without a sampler match or entry.

    Args:
        df: Data the plan is validated against and whose conditioner values are mapped.
        config: PII replacement configuration to resolve.
        data_config: Data parameters, used for grouping and protected columns.
        time_series: Time-series parameters, used for protected columns.
        discoverer: Plan discoverer for ``auto_discovery``; defaults to the heuristic discoverer.
        enhancer: Plan enhancer; defaults to the LLM enhancer configured by ``config.llm``.
        mapping_discoverer: Value-mapping discoverer; defaults to the LLM enhancer configured by ``config.llm``.
        sampler_value_catalog: Sampler values per entity type; loaded from the configured sampler when omitted.
        check_pattern_coverage: Require each pattern to cover at least 99% of ``df``'s values; see
            ``validate_plan``.

    Returns:
        A copy of ``config`` whose ``replacement_plan`` is the resolved inline plan,
        including its complete ``data_to_sampler_value_mapping``, whose
        ``sampler`` is the one used, and whose ``seed`` is set, drawn at random
        when ``config`` left it unset.

    When both plan enhancement and mapping discovery use the default LLM, they share one inference
    session, so a managed local vLLM server starts once and stops before this returns. Plan
    validation runs inside that session, between the two LLM steps.

    Raises:
        ParameterError: If the plan is invalid; for a discovered plan, if the
            mapping does not match the sampler, a conditioner value has no
            sampler match or mapping entry and cannot be discovered, or the plan
            maps ethnicity values while Nemotron-Personas data is missing; or if
            the sampler is set to Nemotron-Personas and its dataset is missing.

    Example:
        With a plan where ``first_name`` depends on ``sex``, values ``"F"`` and
        ``"Woman"`` resolve to ``{"sex": {"F": ["female"], "Woman": ["female"]}}``
        without an LLM, while a value such as ``"X"`` needs a mapping entry.
    """
    if config.is_auto_discovery and enhancer is None and mapping_discoverer is None:
        from ..local_inference import planning_inference_settings
        from .llm import LLMPlanEnhancer

        with planning_inference_settings(config.llm) as settings:
            llm = LLMPlanEnhancer(config.llm, settings=settings)
            return _resolve_replacement_config(
                df,
                config,
                data_config,
                time_series,
                discoverer=discoverer,
                enhancer=llm,
                mapping_discoverer=llm,
                sampler_value_catalog=sampler_value_catalog,
                check_pattern_coverage=check_pattern_coverage,
            )
    return _resolve_replacement_config(
        df,
        config,
        data_config,
        time_series,
        discoverer=discoverer,
        enhancer=enhancer,
        mapping_discoverer=mapping_discoverer,
        sampler_value_catalog=sampler_value_catalog,
        check_pattern_coverage=check_pattern_coverage,
    )

row_counter_columns(df, *, group_column=None)

Return columns whose values number the rows, so automatic discovery does not treat them as PII.

A row counter holds unique, non-missing integers that are exactly 0 to n - 1 or 1 to n for n rows, in any order. Examples include the Unnamed: 0 column pandas writes for a saved index, and plain row_id or event_id sequences. Detection uses the values only, not the column name. The grouping column is never a row counter: it identifies the entity each row belongs to.

Example

A column holding 3, 0, 2, 1 in a four-row dataframe is a row counter; 0, 1, 3, 4 and 17, 18, 19, 20 are not.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/resolver.py
def row_counter_columns(df: pd.DataFrame, *, group_column: str | None = None) -> frozenset[str]:
    """Return columns whose values number the rows, so automatic discovery does not treat them as PII.

    A row counter holds unique, non-missing integers that are exactly ``0`` to
    ``n - 1`` or ``1`` to ``n`` for ``n`` rows, in any order. Examples include
    the ``Unnamed: 0`` column pandas writes for a saved index, and plain
    ``row_id`` or ``event_id`` sequences. Detection uses the values only, not
    the column name. The grouping column is never a row counter: it identifies
    the entity each row belongs to.

    Example:
        A column holding ``3, 0, 2, 1`` in a four-row dataframe is a row
        counter; ``0, 1, 3, 4`` and ``17, 18, 19, 20`` are not.
    """
    return frozenset(str(column) for column in df.columns if column != group_column and _is_row_counter(df[column]))