llm_client
llm_client
¶
Shared OpenAI-compatible client for PII LLM operations.
Classes:
| Name | Description |
|---|---|
InvalidRequestSettingsError |
|
InferenceSettings |
Resolved runtime settings for the OpenAI-compatible adapter. |
TransientInferenceError |
Retryable inference transport failure without response content. |
InvalidInferenceResponse |
Retryable malformed inference envelope without response content. |
MissingInferenceModelError |
An explicit endpoint is configured, but no model ID for it. |
LLMTransport |
Transport capable of requesting a structured LLM response. |
OpenAICompatibleTransport |
Minimal privacy-preserving OpenAI chat-completions transport. |
Functions:
| Name | Description |
|---|---|
nonblank |
Return |
is_loopback_host |
Return whether |
merge_request_options |
Merge request-option layers field by field, dropping fields set to |
reserved_request_fields |
Return the sorted fields in |
configured_model_id |
Return |
resolve_inference_settings |
Resolve settings for an explicit inference endpoint from configuration and environment. |
endpoint_inference_settings |
Return settings for the explicit endpoint in |
Attributes:
| Name | Type | Description |
|---|---|---|
ENDPOINT_ENV |
OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server. |
|
KEY_ENV |
API key for the endpoint. |
|
MODEL_ENV |
Model ID when |
|
REQUEST_OPTIONS_ENV |
Extra chat-completions fields, as a JSON object. |
|
RESERVED_REQUEST_FIELDS |
Chat-completions fields NSS sets itself and request options must not override. |
|
TIMEOUT_ENV |
Per-request timeout in seconds. |
ENDPOINT_ENV = 'NSS_INFERENCE_ENDPOINT'
module-attribute
¶
OpenAI-compatible inference endpoint; when set, planning sends requests there instead of a local server.
KEY_ENV = 'NSS_INFERENCE_KEY'
module-attribute
¶
API key for the endpoint.
MODEL_ENV = 'NSS_INFERENCE_MODEL'
module-attribute
¶
Model ID when replace_pii.llm.model_id is unset.
REQUEST_OPTIONS_ENV = 'NSS_INFERENCE_REQUEST_OPTIONS'
module-attribute
¶
Extra chat-completions fields, as a JSON object.
RESERVED_REQUEST_FIELDS = frozenset({'messages', 'model', 'response_format', 'stream'})
module-attribute
¶
Chat-completions fields NSS sets itself and request options must not override.
TIMEOUT_ENV = 'NSS_INFERENCE_TIMEOUT'
module-attribute
¶
Per-request timeout in seconds.
InvalidRequestSettingsError
¶
Bases: ParameterError
NSS_INFERENCE_TIMEOUT or NSS_INFERENCE_REQUEST_OPTIONS is invalid.
InferenceSettings(endpoint_url, model_id, max_workers, timeout_seconds=DEFAULT_NSS_INFERENCE_TIMEOUT_SECONDS, request_options=dict(), api_key=None)
dataclass
¶
Resolved runtime settings for the OpenAI-compatible adapter.
Attributes:
| Name | Type | Description |
|---|---|---|
endpoint_url |
str
|
Chat-completions base URL without a trailing slash. |
model_id |
str
|
Model identifier sent with every request. |
max_workers |
int
|
Maximum concurrent requests. |
timeout_seconds |
float
|
Per-request timeout; reasoning models on local GPUs can need several minutes. |
request_options |
Mapping[str, object]
|
Extra chat-completions fields, such as sampling settings, sent with every request. |
api_key |
str | None
|
Bearer credential, or |
endpoint_url
instance-attribute
¶
Chat-completions base URL without a trailing slash.
model_id
instance-attribute
¶
Model identifier sent with every request.
max_workers
instance-attribute
¶
Maximum concurrent requests.
timeout_seconds = DEFAULT_NSS_INFERENCE_TIMEOUT_SECONDS
class-attribute
instance-attribute
¶
Per-request timeout; reasoning models on local GPUs can need several minutes.
request_options = field(default_factory=dict)
class-attribute
instance-attribute
¶
Extra chat-completions fields, such as sampling settings, sent with every request.
Fields left out, including temperature, use the server's defaults; vLLM
takes them from the model's generation_config.json.
api_key = field(default=None, repr=False)
class-attribute
instance-attribute
¶
Bearer credential, or None for keyless local endpoints; excluded from repr.
TransientInferenceError(message, *, retry_after=None)
¶
Bases: GenerationError
Retryable inference transport failure without response content.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Description of the failure; must not contain response bodies. |
required |
retry_after
|
float | None
|
Server-requested delay in seconds from a numeric
|
None
|
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
InvalidInferenceResponse
¶
Bases: GenerationError
Retryable malformed inference envelope without response content.
MissingInferenceModelError
¶
Bases: ParameterError
An explicit endpoint is configured, but no model ID for it.
LLMTransport
¶
OpenAICompatibleTransport(settings)
¶
Minimal privacy-preserving OpenAI chat-completions transport.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
settings
|
InferenceSettings
|
Resolved endpoint, model, credential, timeout, and request options. |
required |
Raises:
| Type | Description |
|---|---|
ParameterError
|
If the endpoint is malformed, embeds credentials, or uses plaintext HTTP for a non-loopback host. |
Methods:
| Name | Description |
|---|---|
complete |
Return one structured assistant response. |
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
complete(*, messages, response_model)
¶
Return one structured assistant response.
Errors intentionally exclude response bodies because they may contain prompts, samples, or model-authored sensitive text.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
messages
|
Sequence[Mapping[str, str]]
|
Chat messages to send. |
required |
response_model
|
type[BaseModel]
|
Pydantic model whose strict JSON schema constrains the response. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The assistant message content. |
Raises:
| Type | Description |
|---|---|
ParameterError
|
On authentication, authorization, or other permanent request rejection. |
TransientInferenceError
|
On network failures, timeouts, or retryable HTTP statuses. |
InvalidInferenceResponse
|
If the response envelope is malformed. |
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
nonblank(value)
¶
Return value stripped of surrounding whitespace, or None if blank.
is_loopback_host(hostname)
¶
Return whether hostname is localhost or a loopback IP address.
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
merge_request_options(*layers)
¶
Merge request-option layers field by field, dropping fields set to None.
Later layers win, and a None (JSON null) value removes the field,
leaving it to the server's default.
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
reserved_request_fields(fields)
¶
Return the sorted fields in fields that NSS sets itself.
configured_model_id(config, environment)
¶
Return config.model_id, else NSS_INFERENCE_MODEL, or None when neither is set.
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
resolve_inference_settings(config, *, environ=None)
¶
Resolve settings for an explicit inference endpoint from configuration and environment.
There are no defaults: an endpoint and its model must both be set. The
model resolves from config.model_id, then NSS_INFERENCE_MODEL. As
with other overlapping YAML and environment settings, an explicit YAML
value wins and the environment only supplies its default. The CLI applies
an explicit --inference-model-id to config.model_id before this runs.
The endpoint and API key are deliberately absent from persisted
configuration. They come only from NSS_INFERENCE_ENDPOINT and
NSS_INFERENCE_KEY, which the CLI populates from its runtime options.
The per-request timeout comes from NSS_INFERENCE_TIMEOUT (default 60
seconds) and extra request fields from NSS_INFERENCE_REQUEST_OPTIONS;
NSS sends no sampling settings of its own, so fields left out use the
server's defaults. Plan discovery reaches this through
resolve_planning_inference, which selects the managed local server
instead when no endpoint is set.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
LLMConfig
|
Persisted LLM behavior from |
required |
environ
|
Mapping[str, str] | None
|
Environment mapping to read; defaults to |
None
|
Returns:
| Type | Description |
|---|---|
InferenceSettings
|
Settings with a validated endpoint and resolved model. |
Raises:
| Type | Description |
|---|---|
InvalidRequestSettingsError
|
If the timeout or request options are invalid. |
ParameterError
|
If no endpoint is set, or the endpoint is not an absolute HTTP(S) URL, embeds credentials, or uses plaintext HTTP for a non-loopback host. |
MissingInferenceModelError
|
If the endpoint has no model ID. |
Source code in src/nemo_safe_synthesizer/pii_replacer/llm_client.py
endpoint_inference_settings(config, environment)
¶
Return settings for the explicit endpoint in environment; see resolve_inference_settings.