Skip to content

validation

validation

Dataframe-aware validation for final PII replacement plans.

Functions:

Name Description
get_protected_columns

Return ordering columns that automatic replacement must preserve.

value_matches_pattern

Return whether value follows pattern under entity_type's pattern grammar.

column_pattern_syntax_issue

Return the grammar issue with a replacement column's pattern, independent of any data.

column_pattern_issue

Return the dataframe-aware pattern issue for one replacement column.

identifier_mask_capacity_issue

Return an issue when an identifier mask is too small to keep a distinct column's replacements distinct.

validate_plan

Validate the selected final plan against the dataframe and data configuration.

get_protected_columns(data_config, time_series=None)

Return ordering columns that automatic replacement must preserve.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def get_protected_columns(
    data_config: DataParameters,
    time_series: TimeSeriesParameters | None = None,
) -> frozenset[str]:
    """Return ordering columns that automatic replacement must preserve."""
    candidates = {
        data_config.order_training_examples_by,
        time_series.timestamp_column if time_series is not None else None,
    }
    return frozenset(column for column in candidates if column is not None)

value_matches_pattern(entity_type, pattern, value)

Return whether value follows pattern under entity_type's pattern grammar.

Patterns without a matcher accept every value.

Example

value_matches_pattern(EntityType.UNIQUE_IDENTIFIER, "[aB]", "A") is False.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def value_matches_pattern(entity_type: EntityType, pattern: str, value: str) -> bool:
    """Return whether ``value`` follows ``pattern`` under ``entity_type``'s pattern grammar.

    Patterns without a matcher accept every value.

    Example:
        ``value_matches_pattern(EntityType.UNIQUE_IDENTIFIER, "[aB]", "A")`` is ``False``.
    """
    if ENTITY_BY_TYPE[entity_type].pattern_syntax is PatternSyntax.STRFTIME:
        return _parses_datetime(value, pattern)
    matcher, _ = _pattern_matcher(entity_type, pattern)
    return matcher is None or matcher.fullmatch(value) is not None

column_pattern_syntax_issue(spec)

Return the grammar issue with a replacement column's pattern, independent of any data.

Parameters:

Name Type Description Default
spec PiiColumnPlan

Replacement column carrying the pattern to check.

required

Returns:

Type Description
str | None

A description of the grammar problem, or None when the column has no

str | None

pattern or the pattern follows its entity's grammar.

Raises:

Type Description
InternalError

If the entity's pattern syntax has no matcher.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def column_pattern_syntax_issue(spec: PiiColumnPlan) -> str | None:
    """Return the grammar issue with a replacement column's pattern, independent of any data.

    Args:
        spec: Replacement column carrying the pattern to check.

    Returns:
        A description of the grammar problem, or ``None`` when the column has no
        pattern or the pattern follows its entity's grammar.

    Raises:
        InternalError: If the entity's pattern syntax has no matcher.
    """
    pattern = spec.pattern
    if pattern is None:
        return None
    pattern_syntax = ENTITY_BY_TYPE[spec.entity_type].pattern_syntax
    if pattern_syntax is PatternSyntax.STRFTIME:
        if error := _strftime_pattern_error(pattern):
            return f"column {spec.column_name!r}: pattern {pattern!r} is not valid strftime ({error})"
        return None
    matcher, error = _pattern_matcher(spec.entity_type, pattern)
    if error is not None:
        return f"column {spec.column_name!r}: pattern {pattern!r} {error}"
    if matcher is None:
        raise InternalError(
            f"Pattern syntax {pattern_syntax!r} for entity_type {spec.entity_type.value!r} has no matcher"
        )
    return None

column_pattern_issue(df, spec)

Return the dataframe-aware pattern issue for one replacement column.

A pattern must follow its entity's grammar and cover at least MIN_PATTERN_COVERAGE of the column's non-null values. Columns without a pattern, or absent from df, have no pattern issue.

Parameters:

Name Type Description Default
df DataFrame

Dataframe whose column values the pattern must describe.

required
spec PiiColumnPlan

Replacement column carrying the pattern to check.

required

Returns:

Type Description
str | None

A description of the first problem found, or None when the pattern is valid.

Raises:

Type Description
InternalError

If the entity's pattern syntax has no matcher.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def column_pattern_issue(df: pd.DataFrame, spec: PiiColumnPlan) -> str | None:
    """Return the dataframe-aware pattern issue for one replacement column.

    A pattern must follow its entity's grammar and cover at least
    ``MIN_PATTERN_COVERAGE`` of the column's non-null values. Columns without a
    pattern, or absent from ``df``, have no pattern issue.

    Args:
        df: Dataframe whose column values the pattern must describe.
        spec: Replacement column carrying the pattern to check.

    Returns:
        A description of the first problem found, or ``None`` when the pattern is valid.

    Raises:
        InternalError: If the entity's pattern syntax has no matcher.
    """
    if (issue := column_pattern_syntax_issue(spec)) is not None:
        return issue
    coverage = _pattern_coverage(df, spec)
    if coverage is not None and coverage < MIN_PATTERN_COVERAGE:
        return (
            f"column {spec.column_name!r}: pattern {spec.pattern!r} covers {coverage:.1%} of non-null values; "
            f"at least {MIN_PATTERN_COVERAGE:.0%} is required"
        )
    return None

identifier_mask_capacity_issue(df, spec)

Return an issue when an identifier mask is too small to keep a distinct column's replacements distinct.

Replacements must stay distinct only when the column's original values are all distinct. Each also differs from its own original, so the mask needs more values than the column has. Replacement still succeeds with a smaller mask, but some replacements repeat. Automatic planning uses this to drop an LLM-proposed mask in favor of an unpatterned replacement with more room.

Example

Pattern "USER-#" has 10 values; a column holding the 10 distinct values "USER-0" through "USER-9" returns an issue, while a column where values repeat does not.

Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def identifier_mask_capacity_issue(df: pd.DataFrame, spec: PiiColumnPlan) -> str | None:
    """Return an issue when an identifier mask is too small to keep a distinct column's replacements distinct.

    Replacements must stay distinct only when the column's original values are all distinct. Each also
    differs from its own original, so the mask needs more values than the column has. Replacement still
    succeeds with a smaller mask, but some replacements repeat. Automatic planning uses this to drop an
    LLM-proposed mask in favor of an unpatterned replacement with more room.

    Example:
        Pattern ``"USER-#"`` has 10 values; a column holding the 10 distinct values ``"USER-0"`` through
        ``"USER-9"`` returns an issue, while a column where values repeat does not.
    """
    pattern = spec.pattern
    if pattern is None or spec.entity_type not in _DISTINCT_MASK_ENTITY_TYPES or spec.column_name not in df.columns:
        return None
    values = df[spec.column_name].dropna().astype(str).tolist()
    if not all_distinct(values):
        return None
    # Replacements must differ ignoring letter case, so count the values that still differ without it.
    if (size := MaskSpace.from_pattern(pattern).folded_size) > len(values):
        return None
    return (
        f"column {spec.column_name!r}: pattern {pattern!r} has {size} values that differ ignoring case, too few to give each of "
        f"the column's {len(values)} distinct values its own replacement"
    )

validate_plan(df, plan, *, data_config, time_series=None, check_pattern_coverage=True, warn_on_low_pattern_coverage=False)

Validate the selected final plan against the dataframe and data configuration.

This is the resolver's single dataframe-aware validation gate. Pydantic model construction and LLM response parsing are separate, context-free checks performed at their respective seams.

Parameters:

Name Type Description Default
df DataFrame

Data the plan is validated against.

required
plan PiiReplacementPlan

Final replacement plan.

required
data_config DataParameters

Data parameters, used for grouping and protected columns.

required
time_series TimeSeriesParameters | None

Time-series parameters, used for protected columns.

None
check_pattern_coverage bool

Require each pattern to cover at least 99% of the column's values. Pass False when df is a subset of data the plan was already validated against, such as a training split: a subset can fall just below the threshold although the full input met it. Pattern syntax is checked either way.

True
warn_on_low_pattern_coverage bool

Log a warning instead of failing when a pattern covers less than 99%, as for a plan the user supplied.

False
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
def validate_plan(
    df: pd.DataFrame,
    plan: PiiReplacementPlan,
    *,
    data_config: DataParameters,
    time_series: TimeSeriesParameters | None = None,
    check_pattern_coverage: bool = True,
    warn_on_low_pattern_coverage: bool = False,
) -> None:
    """Validate the selected final plan against the dataframe and data configuration.

    This is the resolver's single dataframe-aware validation gate. Pydantic
    model construction and LLM response parsing are separate, context-free
    checks performed at their respective seams.

    Args:
        df: Data the plan is validated against.
        plan: Final replacement plan.
        data_config: Data parameters, used for grouping and protected columns.
        time_series: Time-series parameters, used for protected columns.
        check_pattern_coverage: Require each pattern to cover at least 99% of
            the column's values. Pass ``False`` when ``df`` is a subset of
            data the plan was already validated against, such as a training
            split: a subset can fall just below the threshold although the
            full input met it. Pattern syntax is checked either way.
        warn_on_low_pattern_coverage: Log a warning instead of failing when a
            pattern covers less than 99%, as for a plan the user supplied.
    """
    issues: list[str] = []
    group_column = data_config.group_training_examples_by
    # Replacement consistency is derived from data configuration rather than
    # stored in the reusable plan: configured groups use group consistency;
    # otherwise every record is replaced independently.
    if group_column is not None and group_column not in df.columns:
        issues.append(f"group column {group_column!r} is not present in the dataframe")

    issues.extend(_iter_reference_issues(df, plan, get_protected_columns(data_config, time_series)))
    issues.extend(
        _iter_pattern_issues(
            df, plan, check_coverage=check_pattern_coverage, warn_on_low_coverage=warn_on_low_pattern_coverage
        )
    )

    if issues:
        details = "\n".join(f"  - {issue}" for issue in issues)
        raise ParameterError(f"Invalid PII replacement plan for this dataframe:\n{details}")