Skip to content

profile

profile

Pinned vLLM serving profiles for the managed local PII planning server.

Classes:

Name Description
LocalVllmProfile

Pinned model and vLLM engine settings for one locally served planner model.

Functions:

Name Description
bundled_profile_names

Return the names of profiles shipped with NSS.

bundled_profile_for_model

Return the bundled profile whose served model name is model_id, if any.

load_profile

Load a bundled profile by name, or a profile YAML file by path.

Attributes:

Name Type Description
RESERVED_VLLM_OPTIONS

vllm serve options NSS sets itself or keeps disabled.

RESERVED_VLLM_OPTIONS = frozenset({'--api-key', '--config', '--enable-log-outputs', '--enable-log-requests', '--host', '--model', '--port', '--revision', '--served-model-name', '--tokenizer-revision', '--uds'}) module-attribute

vllm serve options NSS sets itself or keeps disabled.

The launch owns the address and credential, the profile owns the model identity, and request logging stays off because planning prompts contain raw cell samples. --config is reserved because vLLM reads any option, including these, from the config file it names.

LocalVllmProfile pydantic-model

Bases: BaseModel

Pinned model and vLLM engine settings for one locally served planner model.

Profiles describe only the model and engine. The listening address comes from NSS_INFERENCE_ENDPOINT or a free loopback port, and every launch generates its own API key.

Config:

  • extra: forbid
  • frozen: True

Fields:

Validators:

model_id pydantic-field

Hugging Face model repository or local model directory.

revision pydantic-field

Pinned model revision, preferably a full commit SHA.

served_model_name = None pydantic-field

Model name exposed by the server. Defaults to model_id.

gpu_memory_utilization = 0.9 pydantic-field

Fraction of GPU memory vLLM may reserve. Must be in (0, 1].

max_model_len = None pydantic-field

Maximum prompt plus completion tokens per request. Uses the model's limit when unset.

max_num_seqs = None pydantic-field

Maximum concurrently scheduled requests. Uses the vLLM default when unset.

tensor_parallel_size = 1 pydantic-field

Number of GPUs used for tensor parallelism.

min_gpu_memory_gib = None pydantic-field

Total memory, in GiB, each of the tensor_parallel_size GPUs must have. NSS checks it before downloading weights or launching the server; unset skips the check.

extra_args = () pydantic-field

Additional vllm serve options. Options NSS manages, such as --host, --port, --api-key, and --enable-log-requests, are rejected.

request_options pydantic-field

Chat-completions fields sent with every planning request, such as temperature, top_p, top_k, and thinking_token_budget. Fields left out use the server's defaults. NSS_INFERENCE_REQUEST_OPTIONS overrides individual fields; a null value drops a field.

environment pydantic-field

Extra environment variables for the vLLM server process, such as VLLM_USE_V2_MODEL_RUNNER=0, which the bundled profiles set so thinking_token_budget takes effect.

request_timeout_seconds = DEFAULT_NSS_INFERENCE_TIMEOUT_SECONDS pydantic-field

Per-request timeout for planning calls to this model, including reasoning. NSS_INFERENCE_TIMEOUT overrides it.

startup_timeout_seconds = 600 pydantic-field

Maximum seconds to wait for the server to become ready, including any model download.

shutdown_timeout_seconds = 30 pydantic-field

Seconds to wait after SIGTERM before force-killing the server.

served_name property

Model name clients must send; served_model_name or model_id.

bundled_profile_names()

Return the names of profiles shipped with NSS.

Source code in src/nemo_safe_synthesizer/pii_replacer/local_inference/profile.py
def bundled_profile_names() -> tuple[str, ...]:
    """Return the names of profiles shipped with NSS."""
    return tuple(sorted(path.stem for path in _BUNDLED_PROFILE_DIR.glob(f"*{_PROFILE_SUFFIX}")))

bundled_profile_for_model(model_id)

Return the bundled profile whose served model name is model_id, if any.

Source code in src/nemo_safe_synthesizer/pii_replacer/local_inference/profile.py
def bundled_profile_for_model(model_id: str) -> LocalVllmProfile | None:
    """Return the bundled profile whose served model name is ``model_id``, if any."""
    for name in bundled_profile_names():
        profile = load_profile(name)
        if profile.served_name == model_id:
            return profile
    return None

load_profile(reference)

Load a bundled profile by name, or a profile YAML file by path.

Parameters:

Name Type Description Default
reference str | Path

Bundled profile name, such as "gpt-oss-120b", or a path to a profile YAML file.

required

Returns:

Type Description
LocalVllmProfile

The validated profile.

Raises:

Type Description
ParameterError

If the reference matches neither a bundled profile nor a file, or the file is not a valid profile.

Source code in src/nemo_safe_synthesizer/pii_replacer/local_inference/profile.py
def load_profile(reference: str | Path) -> LocalVllmProfile:
    """Load a bundled profile by name, or a profile YAML file by path.

    Args:
        reference: Bundled profile name, such as ``"gpt-oss-120b"``, or a path
            to a profile YAML file.

    Returns:
        The validated profile.

    Raises:
        ParameterError: If the reference matches neither a bundled profile nor
            a file, or the file is not a valid profile.
    """
    if isinstance(reference, str) and reference in bundled_profile_names():
        path = _BUNDLED_PROFILE_DIR / f"{reference}{_PROFILE_SUFFIX}"
    else:
        path = Path(reference).expanduser()
        if not path.is_file():
            raise ParameterError(
                f"Local inference profile {str(reference)!r} is neither a bundled profile "
                f"({', '.join(bundled_profile_names())}) nor an existing file"
            )

    try:
        data = yaml.safe_load(path.read_text(encoding="utf-8"))
    except (OSError, yaml.YAMLError) as exc:
        raise ParameterError(f"Could not read local inference profile {str(reference)!r}") from exc
    if not isinstance(data, dict):
        raise ParameterError(f"Local inference profile {str(reference)!r} must be a YAML mapping")
    try:
        return LocalVllmProfile.model_validate(data)
    except ValidationError as exc:
        raise ParameterError(f"Invalid local inference profile {str(reference)!r}: {exc}") from exc