sampler_catalog
sampler_catalog
¶
Sampler value catalogs used by dependency-mapping discovery.
Functions:
| Name | Description |
|---|---|
gender_sampler_value |
Return |
nemotron_personas_dataset_path |
Return where the Nemotron-Personas dataset for |
require_nemotron_personas_dataset |
Return the Nemotron-Personas dataset for the configured locale, or raise when it is missing. |
load_sampler_value_catalog |
Return the dependency values understood by the selected sampler. |
available_parquet_columns |
Return the column names in a parquet file's schema without reading its data. |
Attributes:
| Name | Type | Description |
|---|---|---|
PERSONA_SEX_VALUES |
Mapping[str, str]
|
Casefolded Nemotron-Personas |
PERSONA_SEX_VALUES = {'female': 'female', 'male': 'male', '女': 'female', '男': 'male', '여자': 'female', '남자': 'male', 'महिला': 'female', 'पुरुष': 'male', 'mahila': 'female', 'purush': 'male', 'femme': 'female', 'homme': 'male', 'weiblich': 'female', 'männlich': 'male', 'feminino': 'female', 'masculino': 'male'}
module-attribute
¶
Casefolded Nemotron-Personas sex values by locale and the gender each one means.
Only the English locales use Female and Male; ja_JP uses 女, fr_FR Femme, de_DE Weiblich, and so on.
Every sampler accepts female and male, so a value mapping need not depend on the locale.
gender_sampler_value(value)
¶
Return female or male for a Nemotron-Personas sex value, else value casefolded.
Example
Femme and female both give female; a value missing from PERSONA_SEX_VALUES is kept.
Source code in src/nemo_safe_synthesizer/pii_replacer/sampler_catalog.py
nemotron_personas_dataset_path(sampler, locale)
¶
Return where the Nemotron-Personas dataset for locale is read from, whether or not it exists.
Source code in src/nemo_safe_synthesizer/pii_replacer/sampler_catalog.py
require_nemotron_personas_dataset(sampler, locale)
¶
Return the Nemotron-Personas dataset for the configured locale, or raise when it is missing.
Selecting the Nemotron-Personas sampler requires its data; NSS does not
silently substitute Faker here. An unset replace_pii.sampler switches
to Faker before this check; see resolve_replacement_config.
Raises:
| Type | Description |
|---|---|
ParameterError
|
If the locale's dataset file does not exist. |
Source code in src/nemo_safe_synthesizer/pii_replacer/sampler_catalog.py
load_sampler_value_catalog(sampler, locale)
¶
Return the dependency values understood by the selected sampler.
Nemotron-Personas gender values include female and male as well as the locale's own words, such as
femme and homme in fr_FR, so a mapping written for Faker also works with personas.
Raises:
| Type | Description |
|---|---|
ParameterError
|
If the Nemotron-Personas dataset is missing or cannot be read. |
Source code in src/nemo_safe_synthesizer/pii_replacer/sampler_catalog.py
available_parquet_columns(path)
¶
Return the column names in a parquet file's schema without reading its data.