library_builder
library_builder
¶
Executable pipeline for Safe Synthesizer.
Classes:
| Name | Description |
|---|---|
SafeSynthesizer |
Fluent builder and runner for Safe Synthesizer workflows. |
SafeSynthesizer(config=None, workdir=None, save_path=None, emit_telemetry=None, deployment_type=None, unknown_fields=None)
¶
Bases: ConfigBuilder
Fluent builder and runner for Safe Synthesizer workflows.
Extends ConfigBuilder with artifact management and stepwise
pipeline execution. Run all at once via run(), or step by
step::
builder = SafeSynthesizer().with_data_source(df)
builder.process_data().train().generate().evaluate()
builder.save_results()
results = builder.results
train() uses HuggingFaceBackend. generate() chooses
TimeseriesBackend when config.time_series.is_timeseries is true and
VllmBackend otherwise. Stepwise callers must call save_results()
themselves after evaluate(); run() does this automatically.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
SafeSynthesizerParameters | None
|
Optional pre-built parameters that seed every config section. |
None
|
workdir
|
Workdir | None
|
Explicit artifact directory layout. When |
None
|
save_path
|
Path | str | None
|
Root directory for artifacts when |
None
|
Example::
builder = (
SafeSynthesizer()
.with_data_source(df)
.with_replace_pii()
.with_train(learning_rate=0.0001)
.with_generate(num_records=10000)
)
builder.run()
results = builder.results
Methods:
| Name | Description |
|---|---|
load_from_save_path |
Load the Safe Synthesizer configuration from the save path. |
plan_pii_replacement |
Resolve a PII plan and its dependency mappings from the full input dataframe. |
replace_pii |
Replace PII in the complete configured input dataframe. |
process_data |
Perform train/test split, auto-config resolution, and optional PII replacement. |
train |
Fine-tune the base model on the processed training data. |
generate |
Generate synthetic data using the trained model. |
evaluate |
Run quality and privacy evaluations and populate |
run |
Run the full pipeline and save results. |
save_results |
Save synthetic data, evaluation report, and metrics to the workdir. |
Attributes:
| Name | Type | Description |
|---|---|---|
trainer |
TrainingBackend
|
Training backend instance, populated after |
generator |
GeneratorBackend
|
Generation backend instance, populated after |
evaluator |
Evaluator | None
|
Evaluator instance, populated after |
results |
SafeSynthesizerResults
|
Final pipeline results, populated after |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
trainer
instance-attribute
¶
Training backend instance, populated after train().
generator
instance-attribute
¶
Generation backend instance, populated after generate().
evaluator
instance-attribute
¶
Evaluator instance, populated after evaluate() when evaluation is enabled.
results
instance-attribute
¶
Final pipeline results, populated after evaluate() or run().
load_from_save_path(runtime_config=None)
¶
Load the Safe Synthesizer configuration from the save path.
Loads the configuration from the source run directory's config file.
When resuming from a trained model for generation, the source paths
point to the parent workdir that contains the trained adapter.
Optional runtime_config values for generation, evaluation,
telemetry, and the unknown-field policy are applied without mutating
the persisted train config. An explicitly set unknown_fields policy
also controls validation of the saved config itself so legacy fields
can be ignored for version-skew compatibility.
Always prefers cached train/test splits from the training run to ensure evaluation metrics are consistent and privacy guarantees are maintained. Falls back to with_data_source() data only if cached files are missing.
Returns:
| Type | Description |
|---|---|
SafeSynthesizer
|
Self for method chaining. |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 | |
plan_pii_replacement(output_path=None)
¶
Resolve a PII plan and its dependency mappings from the full input dataframe.
This plan-only workflow does not split the input or enter model
metadata, training, generation, replacement, or evaluation stages.
When output_path is provided, the complete NSS configuration is
written there, and its replace_pii.replacement_plan, the part to
review, is written to pii_replacement_plan.yaml in the same directory;
the configuration names that file, so either can be reused.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_path
|
Path | str | None
|
Optional destination for the reusable NSS configuration YAML. |
None
|
Returns:
| Type | Description |
|---|---|
ReplacePiiConfig
|
The dataframe-aware resolved PII replacement configuration. |
Raises:
| Type | Description |
|---|---|
ParameterError
|
If PII replacement is disabled, no dataframe data source is
configured, or |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
replace_pii(output_path=None, *, config_output_path=None, source_text=None)
¶
Replace PII in the complete configured input dataframe.
This standalone workflow resolves planning and sampler mappings against the full input and replaces that same dataframe. The end-to-end pipeline instead resolves against the full input and replaces only its training split after holdout creation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_path
|
Path | str | None
|
Optional destination for the replaced data, written as
Parquet when it ends in |
None
|
config_output_path
|
Path | str | None
|
Optional destination for the complete reusable
configuration. Its |
None
|
source_text
|
DataFrame | None
|
Optional copy of the input with every cell as the
text in its source file, such as |
None
|
Returns:
| Type | Description |
|---|---|
TransformResult
|
The replacement result, including aggregate statistics and the |
TransformResult
|
transformed dataframe. |
Raises:
| Type | Description |
|---|---|
ParameterError
|
If PII replacement is disabled or invalid, no
dataframe data source is configured, or |
Example::
nss = (
SafeSynthesizer()
.with_data_source("patients.csv")
.with_replace_pii(
replacement_plan={"columns_to_replace": [{"column_name": "email", "entity_type": "email"}]},
)
)
# Both paths are optional; without them nothing is written and only the result is returned.
result = nss.replace_pii(
"patients_pii_replaced.csv",
config_output_path="safe-synthesizer-config.yaml",
)
result.transformed_df # Emails replaced; other columns unchanged.
result.generation_statistics.generated_replacement_count # Distinct replacements generated.
result.resolved_config # The plan and sampler mappings written to the YAML file.
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 | |
process_data(check_only=False)
¶
Perform train/test split, auto-config resolution, and optional PII replacement.
Validates configured grouping/ordering columns against the input
dataset, splits the data via Holdout, runs
AutoConfigResolver to resolve "auto" parameters, applies
PII replacement to the training set when enabled, and persists the
splits to the workdir.
When check_only is True (the --validate path), PII
replacement is intentionally skipped and CSV writes are elided; a
resolved config YAML is written instead. Preflight therefore sees
the pre-replacement training split, which is a known gap: PII
replacement can change token lengths, so a clean --validate
does not guarantee a full run will pass token-budget checks. See
the "--validate is best-effort" callout in
docs/user-guide/running.md.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
check_only
|
bool
|
If |
False
|
Returns:
| Type | Description |
|---|---|
SafeSynthesizer
|
Self for method chaining. |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 | |
train()
¶
Fine-tune the base model on the processed training data.
Creates the HuggingFace training backend, loads the base model,
and runs fine-tuning. Requires process_data() to have been
called first.
Returns:
| Type | Description |
|---|---|
SafeSynthesizer
|
Self for method chaining. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called after |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
generate()
¶
Generate synthetic data using the trained model.
Selects the appropriate backend (VllmBackend or
TimeseriesBackend), initializes it, and generates
synthetic records.
Returns:
| Type | Description |
|---|---|
SafeSynthesizer
|
Self for method chaining. |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
evaluate()
¶
Run quality and privacy evaluations and populate results.
Returns:
| Type | Description |
|---|---|
SafeSynthesizer
|
Self for method chaining. |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
run(output_file=None)
¶
Run the full pipeline and save results.
Executes process_data -> train -> generate ->
evaluate -> save_results. For step-by-step control,
call the individual methods instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_file
|
Path | str | None
|
Explicit output path for the synthetic data CSV.
Falls back to |
None
|
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called after |
Source code in src/nemo_safe_synthesizer/sdk/library_builder.py
save_results(output_file=None)
¶
Save synthetic data, evaluation report, and metrics to the workdir.
Writes synthetic_data.csv, evaluation_report.html (when
available), and evaluation_metrics.json into the generate
directory. Called automatically by run(). Call explicitly
after stepwise execution
(process_data().train().generate().evaluate()).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_file
|
Path | str | None
|
Explicit output path for the CSV. Falls back
to |
None
|