inference_environment
inference_environment
¶
The NSS_INFERENCE_* runtime settings for LLM-assisted PII planning, read and validated in one place.
These settings never live in a configuration file. SDK users set the environment
variables; the CLI's --inference-* options validate their values with
InferenceEnvironment and write them to the same variables, so planning reads
them the same way either way. This module imports nothing heavy, so the CLI can
use it at startup.
Classes:
| Name | Description |
|---|---|
InvalidRequestSettingsError |
|
InferenceEnvironment |
The |
Functions:
| Name | Description |
|---|---|
nonblank |
Return |
Attributes:
| Name | Type | Description |
|---|---|---|
ENDPOINT_ENV |
OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server. |
|
KEY_ENV |
API key for the endpoint. |
|
MODEL_ENV |
Model ID when |
|
TIMEOUT_ENV |
Per-request timeout in seconds. |
|
REQUEST_OPTIONS_ENV |
Extra chat-completions fields, as a JSON object. |
|
LOCAL_PROFILE_ENV |
Bundled profile name or profile YAML path for the managed local vLLM server. |
|
INFERENCE_ENV_VARS |
Every environment variable |
|
RESERVED_REQUEST_FIELDS |
Chat-completions fields NSS sets itself and request options must not override. |
ENDPOINT_ENV = 'NSS_INFERENCE_ENDPOINT'
module-attribute
¶
OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server.
KEY_ENV = 'NSS_INFERENCE_KEY'
module-attribute
¶
API key for the endpoint.
MODEL_ENV = 'NSS_INFERENCE_MODEL'
module-attribute
¶
Model ID when replace_pii.llm.model_id is unset.
TIMEOUT_ENV = 'NSS_INFERENCE_TIMEOUT'
module-attribute
¶
Per-request timeout in seconds.
REQUEST_OPTIONS_ENV = 'NSS_INFERENCE_REQUEST_OPTIONS'
module-attribute
¶
Extra chat-completions fields, as a JSON object.
LOCAL_PROFILE_ENV = 'NSS_INFERENCE_LOCAL_PROFILE'
module-attribute
¶
Bundled profile name or profile YAML path for the managed local vLLM server.
INFERENCE_ENV_VARS = (ENDPOINT_ENV, KEY_ENV, MODEL_ENV, TIMEOUT_ENV, REQUEST_OPTIONS_ENV, LOCAL_PROFILE_ENV)
module-attribute
¶
Every environment variable InferenceEnvironment reads.
RESERVED_REQUEST_FIELDS = frozenset({'messages', 'model', 'response_format', 'stream'})
module-attribute
¶
Chat-completions fields NSS sets itself and request options must not override.
InvalidRequestSettingsError
¶
Bases: ParameterError
NSS_INFERENCE_TIMEOUT or NSS_INFERENCE_REQUEST_OPTIONS is invalid.
InferenceEnvironment(endpoint=None, api_key=None, model_id=None, timeout_seconds=None, request_options=dict(), local_profile=None)
dataclass
¶
The NSS_INFERENCE_* settings, with blank values read as unset.
The endpoint is checked where it is used: an explicit endpoint must use HTTPS unless it is a loopback address, while with a local profile it must be the loopback address the managed server listens on.
Example
InferenceEnvironment.from_environ({"NSS_INFERENCE_TIMEOUT": "120"}).timeout_seconds is 120.0.
Methods:
| Name | Description |
|---|---|
from_environ |
Read and validate the settings from |
Attributes:
| Name | Type | Description |
|---|---|---|
request_options |
Mapping[str, object]
|
Chat-completions fields; a |
request_options = field(default_factory=dict)
class-attribute
instance-attribute
¶
Chat-completions fields; a None value drops the field from a lower layer, such as a profile.
from_environ(environ=None)
classmethod
¶
Read and validate the settings from environ, which defaults to os.environ.
Raises:
| Type | Description |
|---|---|
InvalidRequestSettingsError
|
If the timeout is not a positive, finite number of seconds, or the request options are not a JSON object or set a field NSS manages. |
Source code in src/nemo_safe_synthesizer/config/inference_environment.py
nonblank(value)
¶
Return value stripped of surrounding whitespace, or None if blank.