dependency_mappings
dependency_mappings
¶
Resolve dataset dependency columns against sampler value catalogs.
Classes:
| Name | Description |
|---|---|
DependencyMappingInput |
Unmatched values for one dependency column and sampler value catalog. |
Functions:
| Name | Description |
|---|---|
conditioner_columns |
Return each dependency source column and its single resolved entity type. |
mapping_inputs |
Return conditioner values with neither a case-insensitive sampler match nor a mapping entry. |
validate_data_to_sampler_value_mapping |
Validate the plan's value mappings against its conditioner columns and the sampler catalog. |
without_non_categorical_dependencies |
Return |
unmatched_dependency_values |
Count dataset values per conditioner column that have no mapping entry and no sampler match. |
built_in_value_mappings |
Return |
distinct_source_values |
Return distinct values in the form replacement uses to look up mapping keys. |
Attributes:
| Name | Type | Description |
|---|---|---|
BUILT_IN_VALUE_MAPPINGS |
Mapping[EntityType, Mapping[str, str]]
|
Common dataset spellings and the sampler value each maps to, matched ignoring case. |
BUILT_IN_VALUE_MAPPINGS = MappingProxyType({EntityType.GENDER: MappingProxyType({'f': 'female', 'female': 'female', 'woman': 'female', 'girl': 'female', 'm': 'male', 'male': 'male', 'man': 'male', 'boy': 'male'})})
module-attribute
¶
Common dataset spellings and the sampler value each maps to, matched ignoring case.
Resolution writes matching entries into the plan's data_to_sampler_value_mapping
for every sampler, so they are visible and editable in the saved plan. Every
sampler accepts female and male (see PERSONA_SEX_VALUES), so the
entries are the same for every sampler and locale.
DependencyMappingInput(column_name, entity_type, source_values, sampler_values)
dataclass
¶
Unmatched values for one dependency column and sampler value catalog.
Attributes:
| Name | Type | Description |
|---|---|---|
column_name |
str
|
Dependency column the values come from. |
entity_type |
EntityType
|
Entity type the plan assigns to that column. |
source_values |
tuple[str, ...]
|
Distinct dataset values with neither a case-insensitive sampler match nor a mapping entry. |
sampler_values |
tuple[str, ...]
|
Values the sampler accepts for |
column_name
instance-attribute
¶
Dependency column the values come from.
entity_type
instance-attribute
¶
Entity type the plan assigns to that column.
source_values
instance-attribute
¶
Distinct dataset values with neither a case-insensitive sampler match nor a mapping entry.
sampler_values
instance-attribute
¶
Values the sampler accepts for entity_type.
conditioner_columns(plan)
¶
Return each dependency source column and its single resolved entity type.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
mapping_inputs(dataframe, plan, catalog)
¶
Return conditioner values with neither a case-insensitive sampler match nor a mapping entry.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
validate_data_to_sampler_value_mapping(plan, catalog, *, sampler_name)
¶
Validate the plan's value mappings against its conditioner columns and the sampler catalog.
Every entry the sampler cannot use is reported in one error: a target that is not a sampler value, and any mapping for a column whose entity type the sampler has no values for, such as ethnicity with Faker. A plan is checked against the sampler of the run that uses it, so a plan written for another sampler or locale fails here.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
plan
|
PiiReplacementPlan
|
Replacement plan whose |
required |
catalog
|
SamplerValueCatalog
|
Sampler values available for each conditioner entity type. |
required |
sampler_name
|
str
|
The sampler the catalog belongs to, as error messages name it. |
required |
Raises:
| Type | Description |
|---|---|
ParameterError
|
If a mapped column is not a conditioner column, or a mapping entry does not match the sampler. |
Example
With a plan where first_name depends on sex (gender), the
catalog {GENDER: ("female", "male")} accepts
{"sex": {"Woman": ["female"]}} and rejects
{"sex": {"Woman": ["woman"]}}.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
without_non_categorical_dependencies(dataframe, plan, catalog)
¶
Return plan without sampler-conditioned dependencies on non-categorical columns, warning about each.
This is the lenient form of the categorical check in distinct_source_values for a plan the user
supplied. A column with too many distinct values or values too long to map is dropped from every
depends_on and from data_to_sampler_value_mapping, so replacements that depended on it are
drawn without it.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
unmatched_dependency_values(dataframe, plan, catalog)
¶
Count dataset values per conditioner column that have no mapping entry and no sampler match.
Values are compared ignoring case. Columns whose entity type has no sampler catalog are skipped.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataframe
|
DataFrame
|
Data whose conditioner-column values are checked. |
required |
plan
|
PiiReplacementPlan
|
Replacement plan whose |
required |
catalog
|
SamplerValueCatalog
|
Sampler values available for each conditioner entity type. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, int]
|
The count of uncovered distinct values for each column that has any. |
Example
With the catalog {GENDER: ("female", "male")}, a sex column
holding "Female", "Woman" and "X", and the mapping
{"sex": {"Woman": ["female"]}}, the result is {"sex": 1}:
"Female" matches the catalog, "Woman" has an entry, and "X"
has neither.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
built_in_value_mappings(dataframe, plan, catalog)
¶
Return BUILT_IN_VALUE_MAPPINGS entries for dataset values the plan's mapping does not cover.
An entry is added only when the dataset value has no case-insensitive sampler match, the plan's mapping has no entry for it, and its built-in target is in the sampler catalog.
Example
With the catalog {GENDER: ("female", "male")} and a sex column
holding "F", "Female" and "Man", the result is
{"sex": {"F": ["female"], "Man": ["male"]}}. The fr_FR Nemotron-Personas
catalog {GENDER: ("female", "femme", "homme", "male")} gives the same result.
Source code in src/nemo_safe_synthesizer/pii_replacer/planning/dependency_mappings.py
distinct_source_values(values)
¶
Return distinct values in the form replacement uses to look up mapping keys.
Missing values and blank strings are left out: replacement conditions on neither, so they need no mapping entry.
Raises:
| Type | Description |
|---|---|
ParameterError
|
If the column has too many distinct values or values too long to map. |