identifier_space
identifier_space
¶
Finite identifier value spaces with deterministic allocation.
A replacement never equals its own original, and in a column whose originals are all distinct, replacements should stay distinct too. Drawing random values and retrying fails once a value space is nearly full, as with a dense row counter. These spaces instead enumerate their values, so allocation starts at a seeded random position and steps to the next usable value. When a column that must stay distinct has no unused value left, allocation repeats an earlier replacement.
Classes:
| Name | Description |
|---|---|
MaskSpace |
Every value a character mask can produce, in a fixed order. |
NumberSpace |
Non-negative integers below |
IdentifierAllocator |
Hand out replacement values from one space; each differs from its own original. |
Functions:
| Name | Description |
|---|---|
number_space_for |
Return a numeric range for replacing |
identifier_allocator |
Return the allocator for an identifier column, or |
Attributes:
| Name | Type | Description |
|---|---|---|
MASK_ALLOCATED_ENTITY_TYPES |
Entity types whose character-mask pattern defines the whole replacement value space. |
|
NUMBER_ALLOCATED_ENTITY_TYPES |
Entity types whose unpatterned, digit-only values are replaced from a numeric range. |
|
AllocationKind |
How an allocated replacement was chosen: a usable value, or a repeat of an earlier replacement. |
MASK_ALLOCATED_ENTITY_TYPES = frozenset({EntityType.UNIQUE_IDENTIFIER, EntityType.API_KEY})
module-attribute
¶
Entity types whose character-mask pattern defines the whole replacement value space.
Phone numbers are not allocated: a replacement keeps its original's country code and area code, so each original draws from its own part of the mask, and the generator retries to avoid values already taken.
NUMBER_ALLOCATED_ENTITY_TYPES = frozenset({EntityType.UNIQUE_IDENTIFIER, EntityType.API_KEY})
module-attribute
¶
Entity types whose unpatterned, digit-only values are replaced from a numeric range.
AllocationKind = Literal['new', 'repeat']
module-attribute
¶
How an allocated replacement was chosen: a usable value, or a repeat of an earlier replacement.
MaskSpace(positions)
dataclass
¶
Every value a character mask can produce, in a fixed order.
Example
MaskSpace.from_pattern("USER-##") has 100 values, "USER-00"
through "USER-99"; index_of("USER-42") is 42.
Methods:
| Name | Description |
|---|---|
from_pattern |
Parse |
value_at |
Return the value at |
widened |
Return this mask with its last multi-choice position repeated, or |
index_of |
Return the index of |
Attributes:
| Name | Type | Description |
|---|---|---|
positions |
tuple[str | tuple[str, ...], ...]
|
Each mask position: a literal character, or the characters a generated position may produce. |
size |
int
|
Number of distinct values the mask can produce. |
folded_size |
int
|
Number of values that still differ when letter case is ignored. |
positions
instance-attribute
¶
Each mask position: a literal character, or the characters a generated position may produce.
size
property
¶
Number of distinct values the mask can produce.
folded_size
property
¶
Number of values that still differ when letter case is ignored.
Example
MaskSpace.from_pattern("*") has 62 values but only 36 ignoring case.
from_pattern(pattern)
classmethod
¶
Parse pattern as a character mask.
Raises:
| Type | Description |
|---|---|
ParameterError
|
If |
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
value_at(index)
¶
Return the value at index, where 0 <= index < size.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
widened()
¶
Return this mask with its last multi-choice position repeated, or None if it has none.
Only a generated position with more than one allowed character, ignoring letter case, adds room,
since replacements are compared ignoring case. So masks such as ID-[0] and ID-[aA] cannot widen.
Example
MaskSpace.from_pattern("ID-#").widened() produces "ID-00" through "ID-99".
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
index_of(value)
¶
Return the index of value in this space, or None if the mask cannot produce it.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
NumberSpace(digits)
dataclass
¶
Non-negative integers below 10**digits, written without padding.
Example
NumberSpace(3) has the 1,000 values "0" through "999";
index_of("042") and index_of("42.0") are both 42.
Methods:
| Name | Description |
|---|---|
value_at |
Return the value at |
index_of |
Return the integer |
Attributes:
| Name | Type | Description |
|---|---|---|
digits |
int
|
Maximum number of digits in a value. |
size |
int
|
Number of distinct values in the range. |
digits
instance-attribute
¶
Maximum number of digits in a value.
size
property
¶
Number of distinct values in the range.
value_at(index)
¶
index_of(value)
¶
Return the integer value denotes if it is in range, else None.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
IdentifierAllocator(space)
¶
Hand out replacement values from one space; each differs from its own original.
Attributes:
| Name | Type | Description |
|---|---|---|
space |
Value space replacements are drawn from. |
Methods:
| Name | Description |
|---|---|
allocate |
Return a replacement for |
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
free_count
property
¶
Number of values not yet allocated to a column that must stay distinct.
allocate(seed, own_original, *, avoid)
¶
Return a replacement for own_original and how it was chosen, or None if none differs from it.
The replacement is the first value at or after a position drawn uniformly from the whole space
by seed that differs from own_original ignoring letter case: the executor writes each
replacement in its occurrence's case, which would turn "A" back into an original "a". When
avoid is given, the column must stay distinct: the replacement must also not have been
allocated here to a different original or be in avoid, ignoring case. The same original may get
a value again. If no such value is left, a seeded choice among earlier replacements is repeated.
The start is drawn over the whole space rather than taken as seed % size: a seed has 64 bits,
and a long mask such as ADM- followed by 32 alphanumeric positions has far more, so a modulo
would only ever vary the last positions and leave the rest at the first allowed character.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed
|
int
|
Seed that picks the starting position. |
required |
own_original
|
str
|
The original value being replaced. |
required |
avoid
|
Container[str] | None
|
Case-folded values |
required |
Example
In NumberSpace(1), seed 9 starts at 7, so allocate(9, "7", avoid=None) skips "7"
and returns ("8", "new").
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
number_space_for(originals, *, distinct)
¶
Return a numeric range for replacing originals, or None unless they are all integers.
The range covers every integer up to the longest original's digit count. When the replacements must
stay distinct, it gains one digit at a time until it holds at least twice as many values as there
are distinct originals. Originals must be non-negative integers, written with digits only or with a
.0 fraction.
Example
For a unique key "0" through "23485", the result is NumberSpace(5): 100,000 values
for 23,486 originals.
Source code in src/nemo_safe_synthesizer/pii_replacer/replacement/identifier_space.py
identifier_allocator(entity_type, pattern, originals, *, distinct)
¶
Return the allocator for an identifier column, or None if the generator should produce its values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entity_type
|
EntityType
|
The column's entity type. |
required |
pattern
|
str | None
|
The column's character mask, if any. |
required |
originals
|
Sequence[str]
|
Text forms of the column's non-missing original values. |
required |
distinct
|
bool
|
Whether the column's replacements must stay distinct, which sizes a numeric range. |
required |