Configuring Custom Initialization#

The config.py file contains initialization code that runs once at startup, before the LLMRails instance is fully initialized. Use it to register custom providers and set up shared resources.

When to Use config.py or actions.py#

Use Case	File	Reason
Register custom LLM provider	`config.py`	Must happen before LLMRails initialization
Register custom embedding provider	`config.py`	Must happen before LLMRails initialization
Initialize database connection	`config.py`	Shared resource, initialized once
Validate user input	`actions.py`	Called during request processing
Call external API	`actions.py`	Called during request processing
Custom guardrail logic	`actions.py`	Called from Colang flows

Configuration Sections#

Init Function

Define the init() function to initialize resources and register action parameters at startup.

How To

The Init Function

LLM Providers

Register custom text completion and chat models. Use the LLMModel Protocol for the built-in client, or LangChain BaseLLM/BaseChatModel when running with NEMOGUARDRAILS_LLM_FRAMEWORK=langchain.

How To

Custom LLM Providers

Custom LLM Model

Implement the LLMModel protocol to connect a non-OpenAI-compatible backend to the NVIDIA NeMo Guardrails library without depending on LangChain.

How To

Custom LLM Models

Custom LLM Framework

Replace the LLM framework layer to connect LiteLLM, an in-house orchestrator, or any non-default LLM stack to the NVIDIA NeMo Guardrails library.

How To

Custom LLM Frameworks

Embedding Providers

Register custom embedding providers for vector similarity search in the NVIDIA NeMo Guardrails library.