window_group_mia
window_group_mia
¶
Window membership inference protection for grouped numeric time-series data.
The metric asks whether synthetic data sits closer to training sequences than to unseen holdout sequences from the same process, the trace a generator leaves when it memorizes training data. Short windows of consecutive values, across all evaluated numeric columns at once, are compared with their nearest synthetic window. The score measures how much more often training windows are flagged as close than holdout windows are, after removing the difference expected from chance, on a 0--10 scale where 10 means no measurable memorization and 0 means every training window is flagged.
Classes:
| Name | Description |
|---|---|
WindowGroupMIA |
Measure how well synthetic data protects training sequences from window membership inference. |
WindowGroupMIA
pydantic-model
¶
Bases: TimeSeriesComponent
Measure how well synthetic data protects training sequences from window membership inference.
Training and holdout windows are compared with their nearest synthetic
window. A window is flagged when it is closer than all but the closest 5%
of holdout windows. The privacy loss is the log ratio of the training and
holdout flag rates. Chance variation from the binomial counts and from the
holdout groups' own offset is removed before scoring
10 * (exp(-privacy_loss) - alpha) / (1 - alpha), where alpha is the
holdout flag rate, so 10 means equal flag rates and 0 means every training
window is flagged. Exact copies beyond the holdout's own exact-match rate
set a floor on the privacy loss that chance can't remove, so a small
holdout can't hide them. Without a group column or a usable holdout of at
least two groups, the score is unavailable and only copy diagnostics are
reported. With one holdout group, chance alone can cancel the privacy loss
of blurred copies and report a perfect score.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
Display name used in serialized evaluation results. |
details |
dict[str, Any]
|
Flag rates, copy diagnostics, window counts, distances, and per-group summaries. |
Fields:
-
score(EvaluationScore) -
name(str) -
details(dict[str, Any])
name = 'Window Membership Inference Protection'
pydantic-field
¶
Display name used in serialized evaluation results.
details
pydantic-field
¶
Flag rates, copy diagnostics, window counts, distances, and per-group summaries.
jinja_context
cached
property
¶
Return score, details, card attributes, and chart figures for report rendering.
from_evaluation_datasets(evaluation_datasets, config=None)
staticmethod
¶
Evaluate window membership inference protection for paired time-series datasets.
Report orchestration controls whether the optional metric runs.
Calling this component directly always computes it with the supplied
configuration or isolated defaults. Evaluation failures are returned
in score.notes instead of aborting the full evaluation pipeline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
evaluation_datasets
|
EvaluationDatasets
|
Training, synthetic, and holdout ( |
required |
config
|
SafeSynthesizerParameters | None
|
Optional pipeline and metric configuration. |
None
|
Returns:
| Type | Description |
|---|---|
WindowGroupMIA
|
A component containing the score, diagnostic details, and notes. |