validation
validation
¶
Dataframe-aware validation for final PII replacement plans.
Functions:
| Name | Description |
|---|---|
get_protected_columns |
Return ordering columns that automatic replacement must preserve. |
value_matches_pattern |
Return whether |
column_pattern_syntax_issue |
Return the grammar issue with a replacement column's pattern, independent of any data. |
column_pattern_issue |
Return the dataframe-aware pattern issue for one replacement column. |
identifier_mask_capacity_issue |
Return an issue when an identifier mask is too small to keep a distinct column's replacements distinct. |
validate_plan |
Validate the selected final plan against the dataframe and data configuration. |
get_protected_columns(data_config, time_series=None)
¶
Return ordering columns that automatic replacement must preserve.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
value_matches_pattern(entity_type, pattern, value)
¶
Return whether value follows pattern under entity_type's pattern grammar.
Patterns without a matcher accept every value.
Example
value_matches_pattern(EntityType.UNIQUE_IDENTIFIER, "[aB]", "A") is False.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
column_pattern_syntax_issue(spec)
¶
Return the grammar issue with a replacement column's pattern, independent of any data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
PiiColumnPlan
|
Replacement column carrying the pattern to check. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
A description of the grammar problem, or |
str | None
|
pattern or the pattern follows its entity's grammar. |
Raises:
| Type | Description |
|---|---|
InternalError
|
If the entity's pattern syntax has no matcher. |
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
column_pattern_issue(df, spec)
¶
Return the dataframe-aware pattern issue for one replacement column.
A pattern must follow its entity's grammar and cover at least
MIN_PATTERN_COVERAGE of the column's non-null values. Columns without a
pattern, or absent from df, have no pattern issue.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe whose column values the pattern must describe. |
required |
spec
|
PiiColumnPlan
|
Replacement column carrying the pattern to check. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
A description of the first problem found, or |
Raises:
| Type | Description |
|---|---|
InternalError
|
If the entity's pattern syntax has no matcher. |
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
identifier_mask_capacity_issue(df, spec)
¶
Return an issue when an identifier mask is too small to keep a distinct column's replacements distinct.
Replacements must stay distinct only when the column's original values are all distinct. Each also differs from its own original, so the mask needs more values than the column has. Replacement still succeeds with a smaller mask, but some replacements repeat. Automatic planning uses this to drop an LLM-proposed mask in favor of an unpatterned replacement with more room.
Example
Pattern "USER-#" has 10 values; a column holding the 10 distinct values "USER-0" through
"USER-9" returns an issue, while a column where values repeat does not.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/validation.py
validate_plan(df, plan, *, data_config, time_series=None, check_pattern_coverage=True, warn_on_low_pattern_coverage=False)
¶
Validate the selected final plan against the dataframe and data configuration.
This is the resolver's single dataframe-aware validation gate. Pydantic model construction and LLM response parsing are separate, context-free checks performed at their respective seams.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Data the plan is validated against. |
required |
plan
|
PiiReplacementPlan
|
Final replacement plan. |
required |
data_config
|
DataParameters
|
Data parameters, used for grouping and protected columns. |
required |
time_series
|
TimeSeriesParameters | None
|
Time-series parameters, used for protected columns. |
None
|
check_pattern_coverage
|
bool
|
Require each pattern to cover at least 99% of
the column's values. Pass |
True
|
warn_on_low_pattern_coverage
|
bool
|
Log a warning instead of failing when a pattern covers less than 99%, as for a plan the user supplied. |
False
|