Skip to content

inference_environment

inference_environment

The NSS_INFERENCE_* runtime settings for LLM-assisted PII planning, read and validated in one place.

These settings never live in a configuration file. SDK users set the environment variables; the CLI's --inference-* options validate their values with InferenceEnvironment and write them to the same variables, so planning reads them the same way either way. This module imports nothing heavy, so the CLI can use it at startup.

Classes:

Name Description
InvalidRequestSettingsError

NSS_INFERENCE_TIMEOUT or NSS_INFERENCE_REQUEST_OPTIONS is invalid.

InferenceEnvironment

The NSS_INFERENCE_* settings, with blank values read as unset.

Functions:

Name Description
nonblank

Return value stripped of surrounding whitespace, or None if blank.

Attributes:

Name Type Description
ENDPOINT_ENV

OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server.

KEY_ENV

API key for the endpoint.

MODEL_ENV

Model ID when replace_pii.llm.model_id is unset.

TIMEOUT_ENV

Per-request timeout in seconds.

REQUEST_OPTIONS_ENV

Extra chat-completions fields, as a JSON object.

LOCAL_PROFILE_ENV

Bundled profile name or profile YAML path for the managed local vLLM server.

INFERENCE_ENV_VARS

Every environment variable InferenceEnvironment reads.

RESERVED_REQUEST_FIELDS

Chat-completions fields NSS sets itself and request options must not override.

ENDPOINT_ENV = 'NSS_INFERENCE_ENDPOINT' module-attribute

OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server.

KEY_ENV = 'NSS_INFERENCE_KEY' module-attribute

API key for the endpoint.

MODEL_ENV = 'NSS_INFERENCE_MODEL' module-attribute

Model ID when replace_pii.llm.model_id is unset.

TIMEOUT_ENV = 'NSS_INFERENCE_TIMEOUT' module-attribute

Per-request timeout in seconds.

REQUEST_OPTIONS_ENV = 'NSS_INFERENCE_REQUEST_OPTIONS' module-attribute

Extra chat-completions fields, as a JSON object.

LOCAL_PROFILE_ENV = 'NSS_INFERENCE_LOCAL_PROFILE' module-attribute

Bundled profile name or profile YAML path for the managed local vLLM server.

INFERENCE_ENV_VARS = (ENDPOINT_ENV, KEY_ENV, MODEL_ENV, TIMEOUT_ENV, REQUEST_OPTIONS_ENV, LOCAL_PROFILE_ENV) module-attribute

Every environment variable InferenceEnvironment reads.

RESERVED_REQUEST_FIELDS = frozenset({'messages', 'model', 'response_format', 'stream'}) module-attribute

Chat-completions fields NSS sets itself and request options must not override.

InvalidRequestSettingsError

Bases: ParameterError

NSS_INFERENCE_TIMEOUT or NSS_INFERENCE_REQUEST_OPTIONS is invalid.

InferenceEnvironment(endpoint=None, api_key=None, model_id=None, timeout_seconds=None, request_options=dict(), local_profile=None) dataclass

The NSS_INFERENCE_* settings, with blank values read as unset.

The endpoint is checked where it is used: an explicit endpoint must use HTTPS unless it is a loopback address, while with a local profile it must be the loopback address the managed server listens on.

Example

InferenceEnvironment.from_environ({"NSS_INFERENCE_TIMEOUT": "120"}).timeout_seconds is 120.0.

Methods:

Name Description
from_environ

Read and validate the settings from environ, which defaults to os.environ.

Attributes:

Name Type Description
request_options Mapping[str, object]

Chat-completions fields; a None value drops the field from a lower layer, such as a profile.

request_options = field(default_factory=dict) class-attribute instance-attribute

Chat-completions fields; a None value drops the field from a lower layer, such as a profile.

from_environ(environ=None) classmethod

Read and validate the settings from environ, which defaults to os.environ.

Raises:

Type Description
InvalidRequestSettingsError

If the timeout is not a positive, finite number of seconds, or the request options are not a JSON object or set a field NSS manages.

Source code in src/nemo_safe_synthesizer/config/inference_environment.py
@classmethod
def from_environ(cls, environ: Mapping[str, str] | None = None) -> Self:
    """Read and validate the settings from ``environ``, which defaults to ``os.environ``.

    Raises:
        InvalidRequestSettingsError: If the timeout is not a positive, finite number of
            seconds, or the request options are not a JSON object or set a field NSS manages.
    """
    values = os.environ if environ is None else environ
    return cls(
        endpoint=nonblank(values.get(ENDPOINT_ENV)),
        api_key=nonblank(values.get(KEY_ENV)),
        model_id=nonblank(values.get(MODEL_ENV)),
        timeout_seconds=_timeout_seconds(nonblank(values.get(TIMEOUT_ENV))),
        request_options=_request_options(nonblank(values.get(REQUEST_OPTIONS_ENV))),
        local_profile=nonblank(values.get(LOCAL_PROFILE_ENV)),
    )

nonblank(value)

Return value stripped of surrounding whitespace, or None if blank.

Source code in src/nemo_safe_synthesizer/config/inference_environment.py
def nonblank(value: str | None) -> str | None:
    """Return ``value`` stripped of surrounding whitespace, or ``None`` if blank."""
    if value is None:
        return None
    stripped = value.strip()
    return stripped or None