Before You Begin Configuring Rails#

This Configure Rails chapter thoroughly describes how to prepare guardrails configuration files. This page covers the prerequisites and decisions to make before you begin working on guardrails configurations.

Checklist Summary#

Use the following checklist to ensure that you have all the necessary components ready before you begin configuring guardrails.

Each item in the checklist is described in detail in the following sections.

Hosted LLM for the Main LLM#

You need a main LLM hosted and accessible via API. This LLM handles the conversation by generating responses to user queries.

Options:

Provider

Requirements

NVIDIA NIM

Deploy NIM and note the API endpoint

OpenAI

Obtain API key

Azure OpenAI

Configure Azure endpoint and API key

Other providers

Refer to Supported LLMs

Checklist of what you need:

  • [ ] LLM API endpoint URL, either locally, on NVIDIA API Catalog, or on the third-party providers

  • [ ] Authentication credentials (API key or token)

NemoGuard NIM Microservices#

Deploy dedicated safety models to offload guardrail checks from the main LLM:

NemoGuard Model

Purpose

Content Safety

Detect harmful or inappropriate content

Jailbreak Detection

Block adversarial prompt attacks

Topic Control

Keep conversations on-topic

Checklist of what you need:

  • [ ] NemoGuard NIM endpoint URLs, either locally or on NVIDIA API Catalog

  • [ ] KV cache enabled for better performance (recommended)

Tip

If you use NVIDIA NIM for LLMs and LLM-based NemoGuard NIMs, KV cache helps reduce latency for sequential guardrail checks. To learn more about KV cache, see the KV Cache Reuse guide in the NVIDIA NIM documentation.

Knowledge Base Documents#

If using RAG (Retrieval-Augmented Generation) for grounded responses:

  • [ ] Prepare documents in markdown format (.md files)

  • [ ] Organize documents in a kb/ folder

Advanced Components#

For advanced use cases such as implementing your own custom scripts or guardrails, prepare the following as needed:

Component

Purpose

Format

Custom Actions

External API calls, validation logic

Python functions in actions.py

Custom Initialization

Register custom LLM/embedding providers

Python code in config.py

Custom Prompts

Override default guardrails prompts

YAML in config.yml

Next Steps#

Once you have these components ready, proceed to the next section Configuration Overview to start organizing your guardrails configuration files.

If you need tutorials to understand how to use the NeMo Guardrails toolkit, revisit the Get Started section.