Overview of NVIDIA NeMo Guardrails Library

The NVIDIA NeMo Guardrails library (PyPI | GitHub) is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies.

NeMo Guardrails Library within the NVIDIA NeMo Software Stack

NVIDIA NeMo is a suite of microservices, tools, and libraries for building, deploying, and scaling LLM-based applications.

NeMo Guardrails is part of the NVIDIA NeMo software stack. It takes part in adding programmable guardrails to LLM-based applications. The NeMo Guardrails library provides tools to build guardrails and integrate them into your LLM-based applications at development time. The NeMo Guardrails microservice as part of the NeMo microservices platform is a production-ready container image built on top of this library, designed for Kubernetes deployment with Helm charts.

	NeMo Guardrails Library	NeMo Guardrails Microservice
Distribution	PyPI (pip install)	Container image (backed by this library)
Deployment	Self-managed Python environment	Kubernetes with Helm
Scaling	Application-level	Managed by orchestrator
Configuration	YAML + Colang	Same YAML + Colang format

Configurations are portable between the library and microservice, so you can develop locally with the library and deploy to production with the microservice.

Architecture

The NeMo Guardrails library is designed to be integrated into LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies.

        %%{init: {'theme': 'neutral', 'themeVariables': { 'background': 'transparent' }}}%%

flowchart TB
  A("Application Code")
  B("NeMo Guardrails Library")
  C("Large Language Model (LLM)")

  A <--> B

subgraph NemoGuard["NemoGuard NIMs"]
  direction TB
  D("NemoGuard Content Safety")
  E("NemoGuard Topic Control")
  F("NemoGuard Jailbreak Detection")
end

  B <--> NemoGuard
  NemoGuard <--> C

  style A fill:#d8d8e8,stroke:#999
  style B fill:#f0f7e6,stroke:#76b900,stroke-width:2px
  style C fill:#d8d8e8,stroke:#999
  style D fill:#f0f7e6,stroke:#76b900
  style E fill:#f0f7e6,stroke:#76b900
  style F fill:#f0f7e6,stroke:#76b900
    

Application code interacting with LLMs through the NeMo Guardrails library.

---

Use Cases

The following are the top use cases of the NeMo Guardrails library that you can apply to protect your LLM applications.

🛡️ Add Content Safety

Content safety guardrails help ensure that both user inputs and LLM outputs are safe and appropriate. The NeMo Guardrails library provides multiple approaches to content safety:

• LLM self-checking: Use the LLM itself to check inputs and outputs for harmful content.

• NVIDIA safety models: Integrate with Llama 3.1 NemoGuard 8B Content Safety for robust content moderation.

• Community models: Use LlamaGuard, Fiddler Guardrails, and other community content safety solutions.

• Third-party APIs: Integrate with ActiveFence, Cisco AI Defense, and other moderation services.

For practical examples, try the following tutorials:

• Content Safety - Text

• Content Safety - Multimodal

🔒 Add Jailbreak Protection

Jailbreak protection helps prevent adversarial attempts from bypassing safety measures and manipulating the LLM into generating harmful or unwanted content. The NeMo Guardrails library provides multiple layers of jailbreak protection:

• Self-check jailbreak detection: Use the LLM to identify jailbreak attempts.

• Heuristic detection: Use pattern-based detection for common jailbreak techniques.

• NVIDIA NemoGuard: Integrate with NemoGuard Jailbreak Detection NIM for advanced threat detection.

• Third-party integrations: Use Prompt Security, Pangea AI Guard, and other services.

For practical examples, try the following tutorial:

• Detect Jailbreak Attempts with NVIDIA NemoGuard JailbreakDetect NIM

🎯 Control Topic Conversation

Topic control guardrails ensure that conversations stay within predefined subject boundaries and prevent the LLM from engaging in off-topic discussions. This is implemented through:

• Dialog rails: Pre-define conversational flows using the Colang language.

• Topical rails: Control what topics the bot can and cannot discuss.

• NVIDIA NemoGuard: Integrate with NemoGuard Topic Control NIM for semantic topic detection.

For practical examples, try the following tutorial:

• Restrict Topics with Llama 3.1 NemoGuard 8B TopicControl NIM

🔐 Detect and Mask PII

Personally Identifiable Information (PII) detection helps protect user privacy by detecting and masking sensitive data in user inputs, LLM outputs, and retrieved content. The NeMo Guardrails library supports PII detection through multiple integrations:

• Presidio-based detection: Use Microsoft Presidio for detecting entities such as names, email addresses, phone numbers, social security numbers, and more.

• Private AI: Integrate with Private AI for advanced PII detection and masking.

• AutoAlign: Use AutoAlign PII detection with customizable entity types.

• GuardrailsAI: Access GuardrailsAI PII validators from the Guardrails Hub.

PII detection can be configured to either detect and block content containing PII or to mask PII entities before processing.

For more information, refer to the Presidio Integration and Sensitive Data Detection section in the built-in Guardrails library.

🤖 Add Agentic Security

Agentic security provides specialized guardrails for LLM-based agents that use tools and interact with external systems. This includes:

• Tool call validation: Execute rails that validate tool inputs and outputs before and after invocation.

• Agent workflow protection: Integrate with LangGraph for multi-agent safety.

• Secure tool integration: Review guidelines for safely connecting LLMs to external resources (refer to Security Guidelines).

• Action monitoring: Monitor detailed logging and tracing of agent actions.

Key security considerations for agent systems:

1. Isolate all authentication information from the LLM.

2. Validate and sanitize all tool inputs.

3. Apply execution rails to tool calls.

4. Monitor agent behavior for unexpected actions.

For more information, refer to the Tools Integration Guide, Security Guidelines, and LangGraph Integration.

🔧 Build Your Own or Use Third-party Guardrail Solutions

The NeMo Guardrails library provides extensive flexibility for creating custom guardrails tailored to your specific requirements. You can either build your own guardrails or use third-party guardrails. If you have a script or tool that runs a custom guardrail, you can use it in NeMo Guardrails by following one of these approaches:

1. Python actions: Create custom actions in Python for complex logic and external integrations. For more information, refer to the Custom Actions.

2. LangChain tool integration: Register LangChain tools as custom actions. For more information, refer to the Tools Integration with the NeMo Guardrails Library.

3. Third-party API integration: Integrate external moderation and validation services. For a complete list of supported third-party guardrail services, refer to the Third-Party APIs section in the built-in Guardrails library.

🔌 Integrate NeMo Guardrails Library into Your Application

You can integrate the NeMo Guardrails library into your application using the tools provided by the library.

1. Python SDK: Use the Python SDK to add guardrails directly into your Python application.

   from nemoguardrails import LLMRails, RailsConfig
   
   config = RailsConfig.from_path("./config")
   rails = LLMRails(config)
   
   response = rails.generate(
       messages=[{"role": "user", "content": "Hello!"}]
   )
   

   The generate method accepts the same message format as the OpenAI Chat Completions API.

2. Server API: You can solely set up a guardrails server after programming guardrails using the Python SDK. You can then start a local NeMo Guardrails server with the following command. This server exposes an HTTP API compatible with OpenAI’s /v1/chat/completions endpoint.

   nemoguardrails server --config ./config --port 8000
   

   The server exposes HTTP APIs compatible with OpenAI’s /v1/chat/completions endpoint. You can then use the server in your application by sending requests to the server’s endpoint.

---

Tools

The following are the tools you can use to interact with the NeMo Guardrails library.

Python SDK

from nemoguardrails import LLMRails, RailsConfig

config = RailsConfig.from_path("./config")
rails = LLMRails(config)

response = rails.generate(
    messages=[{"role": "user", "content": "Hello!"}]
)

The generate method accepts the same message format as the OpenAI Chat Completions API.

CLI Server

nemoguardrails server --config ./config --port 8000

The server exposes an HTTP API compatible with OpenAI’s /v1/chat/completions endpoint.