NeMo Guardrails Library Troubleshooting Guide#

This page covers common issues you may encounter when configuring, running, or monitoring the NVIDIA NeMo Guardrails library, along with their resolution steps.

Get Help

If your issue is not listed here, open an issue on GitHub.

Runtime#

Nested AsyncIO Loop#

The NVIDIA NeMo Guardrails library is async-first, so the core functionality is implemented with async functions. To provide a blocking API, the library must invoke async functions inside synchronous code with asyncio.run.

Python does not allow nested event loops. In notebooks, async web servers, and other environments that already run an event loop, nested loop behavior can cause runtime errors or unexpected behavior. This issue is being discussed by the Python core team and will likely be supported in the future. For more information, refer to GitHub Issue 66435 and Pull Request 93338.

The NVIDIA NeMo Guardrails library uses nest_asyncio as a workaround. The patching is applied when the nemoguardrails package is loaded the first time.

If you do not need the blocking API, or if the nest_asyncio patching causes unexpected problems, disable it before loading nemoguardrails:

$ export DISABLE_NEST_ASYNCIO=True

Then restart the Python process and retry the application.

LLM Framework Routing#

Starting with 0.22, two controls select whether an engine is handled by the built-in OpenAI-compatible client or by LangChain:

  • NEMOGUARDRAILS_LLM_FRAMEWORK environment variable. Read once at process startup. Default value default (built-in). Set to langchain to opt into the LangChain path, or to any name you register with register_framework(name, instance) before initialization.

  • nemoguardrails.set_default_framework(name). Switches the active selection at runtime. Raises KeyError if the name is unknown and is not one of the lazy built-ins (default, langchain).

Use the environment variable when every model in a deployment uses the same path. Use set_default_framework from Python when you switch dynamically, such as in tests or when bootstrapping a custom framework.

For migration recipes, refer to Migrating to 0.22. For the engine-by-engine matrix, refer to Supported LLMs.

Error: No Default base_url for Provider#

ValueError: No default base_url for provider 'cohere'.
If your endpoint is OpenAI-compatible, set parameters.base_url.
Otherwise, set NEMOGUARDRAILS_LLM_FRAMEWORK=langchain and install
the matching langchain-<provider> package (see migration guide).

This error appears when the engine name you used (cohere in the example) is not a built-in OpenAI-compatible engine and you have not opted into LangChain. The error itself names both fix paths; pick the one that matches your provider. For migration recipes, refer to Migrating to 0.22.

Error: Framework Already Registered#

register_framework does not allow rebinding. If you see this error, the framework name is already registered in the current process.

Pick a different name. In tests, call the registry-reset hook before re-registering the same name.

Error: Unknown Framework#

The set_default_framework call used a name that is not registered and is not one of the lazy built-ins default or langchain.

Register the framework first, or correct the name:

from nemoguardrails import register_framework, set_default_framework
from my_pkg import MyFramework

register_framework("my-framework", MyFramework())
set_default_framework("my-framework")

Error: Unsupported Parameter on First Call#

Starting with 0.22, the built-in client forwards parameters from config.yml directly to the OpenAI-compatible HTTP request. Keys that LangChain accepted as Python flags (streaming, disable_streaming, verbose, cache, callbacks, tags, metadata, name, model_kwargs) and provider-prefixed credential aliases (openai_api_base, nim_base_url, *_api_key, and others) are not part of the OpenAI wire shape, so the provider rejects them. The library detects recognizable shapes at boot and on the first 400/422 response, and appends a migration hint to the underlying provider error.

Fix the configuration by choosing one path:

  • Adapt the configuration to OpenAI-compatible shape. Rename openai_api_base to base_url, drop LangChain Python flags, and remove provider-prefixed aliases. The migration recipe in Migrating to 0.22 covers the common case.

  • Keep the 0.21 config. Set NEMOGUARDRAILS_LLM_FRAMEWORK=langchain for the process and install LangChain plus the matching upstream provider integration. The legacy field names continue to work when you opt into LangChain.

Guardrails Metrics#

No Metrics Appear in Your Backend#

Call set_meter_provider(...) before constructing Guardrails(config, use_iorails=True, require_iorails=True) (or LLMRails(config, require_iorails=True) when running with NEMO_GUARDRAILS_IORAILS_ENGINE=1). Then verify that metrics.enabled: true is set in the configuration. require_iorails=True makes the constructor raise if the config is incompatible with IORails, so metrics misconfiguration surfaces immediately rather than silently disabling metrics.

Metrics Are Silently Missing#

When metrics.enabled: true but no MeterProvider is configured, the OpenTelemetry API returns a no-op meter and silently discards every emission. The library does not log a warning.

Verify locally with ConsoleMetricExporter first. Then ensure set_meter_provider(...) runs before constructing Guardrails(config, use_iorails=True, require_iorails=True) (or LLMRails(config, require_iorails=True) when running with NEMO_GUARDRAILS_IORAILS_ENGINE=1).

Metrics Dependency Is Missing#

If you see the following warning, the opentelemetry-api package is not installed:

UserWarning: Metrics are enabled in config but the opentelemetry-api package is not installed

Install the dependency:

$ pip install "nemoguardrails[tracing]"

Metrics Are Emitted but Never Reach the Backend#

Verify that the exporter target is reachable. Test with ConsoleMetricExporter first to confirm IORails engine emission, then swap in the production exporter.

LLMRails Produces No Metrics#

Metrics are emitted only by the IORails engine. Enable it either by setting NEMO_GUARDRAILS_IORAILS_ENGINE=1 (which redirects LLMRails(config) to the IORails engine) or by constructing Guardrails(config, use_iorails=True) directly, and use generate_async or stream_async. Pass require_iorails=True to make the constructor raise (instead of silently falling back to LLMRails and emitting no metrics) when the config is incompatible with IORails.

Synchronous generate() Produces No Metrics#

Telemetry is disabled for the ephemeral engine constructed by the synchronous generate() shim. Use generate_async or stream_async for production paths.

gen_ai.client.token.usage Is Missing for Streaming Requests#

The upstream provider did not return a usage field in the streamed response. Forward stream_options={"include_usage": true} when calling OpenAI-compatible providers, or accept that token usage is not available for that provider.

Histogram Buckets Are Wrong in the Backend#

The library sets bucket-boundary advisories per the OpenTelemetry spec. Verify that your backend honors the SDK’s explicit_bucket_boundaries_advisory. Some Prometheus exporters override the advisory unless explicitly configured.

guardrails.requests.active Drifts from the Sum of Saturation Gauges#

A small steady drift is expected because the gauge reads are not atomic with the counter increments. A persistent large drift indicates an instrumentation bug. Open an issue.

Wrong service.name on Metrics#

Set the Resource with service.name when constructing the MeterProvider. Use the same Resource on the TracerProvider to keep traces and metrics correlated.