Installation Guide#

This guide walks you through the following steps to install the NeMo Guardrails toolkit.

  1. Check the requirements.

  2. Set up a fresh virtual environment.

  3. Install using pip.

  4. Install from Source Code.

  5. Install optional dependencies.

  6. Use Docker.

Requirements#

Review the following requirements to install the NeMo Guardrails toolkit.

Requirement Type

Details

Hardware

The toolkit runs on CPUs (no GPUs required).
It acts as a process manager between your app front-end and the backend LLM.

Software

Python 3.10, 3.11, 3.12, or 3.13

Additional Dependencies#

The NeMo Guardrails toolkit uses annoy, which is a C++ library with Python bindings. To install it, you need to have a valid C++ runtime on your computer. Most systems already have installed a C++ runtime. If the annoy installation fails due to a missing C++ runtime, you can install a C++ runtime as follows:

Installing a C++ runtime on Linux, Mac, or Unix-based OS#

  1. Install gcc and g++ using apt-get install gcc g++.

  2. Update the following environment variables: export CC=path_to_clang and export CXX=path_to_clang (usually, path_to_clang is /usr/bin/clang).

  3. In some cases, you might also need to install the python-dev package using apt-get install python-dev (or apt-get install python3-dev). Check out this thread if the error persists.

Installing a C++ runtime on Windows#

Install the Microsoft C++ Build Tools. This installs Microsoft Visual C++ (version 14.0 or greater is required by the latest version of annoy).

Setting up a virtual environment#

To experiment with the NeMo Guardrails toolkit from scratch, use a fresh virtual environment. Otherwise, you can skip to the following section.

Setting up a virtual environment on Linux, Mac, or Unix-based OS#

  1. Create a folder, such as my_assistant, for your project.

mkdir my_assistant
cd my_assistant

  1. Create a virtual environment.

python3 -m venv venv

  1. Activate the virtual environment.

source venv/bin/activate

Setting up a virtual environment on Windows#

  1. Open a new CMD prompt (Windows Key + R, cmd.exe)

  2. Install virtualenv using the command pip install virtualenv

  3. Check that virtualenv is installed using the command pip --version.

  4. Install virtualenvwrapper-win using the command pip install virtualenvwrapper-win.

Use the mkvirtualenv name command to activate a new virtual environment called name.

Install the NeMo Guardrails Toolkit#

Install the NeMo Guardrails toolkit using pip:

pip install nemoguardrails

Installing from source code#

The NeMo Guardrails toolkit is under active development and the main branch always contains the latest development version. To install from source:

  1. Clone the repository:

    git clone https://github.com/NVIDIA/NeMo-Guardrails.git

  2. Install the package locally:

    cd NeMo-Guardrails
pip install -e .

Extra dependencies#

The nemoguardrails package also defines the following extra dependencies:

  • dev: packages required by some extra Guardrails features for developers, such as the autoreload feature.

  • eval: packages used for the Guardrails evaluation tools.

  • openai: installs the latest openai package supported by the NeMo Guardrails toolkit.

  • sdd: packages used by the sensitive data detector integrated in the NeMo Guardrails toolkit.

  • all: installs all extra packages.

To keep the footprint of nemoguardrails as small as possible, these are not installed by default. To install any of the extra dependency you can use pip as well. For example, to install the dev extra dependencies, run the following command:

> pip install nemoguardrails[dev]

Optional dependencies#

Warning

If pip fails to resolve dependencies when running pip install nemoguardrails[all], you should specify additional constraints directly in the pip install command.

Example Command:

pip install "nemoguardrails[all]" "pandas>=1.4.0,<3"

To use OpenAI, just use the openai extra dependency that ensures that all required packages are installed. Make sure the OPENAI_API_KEY environment variable is set, as shown in the following example, where YOUR_KEY is your OpenAI key.

pip install nemoguardrails[openai]
export OPENAI_API_KEY=YOUR_KEY

Some NeMo Guardrails toolkit LLMs and features have specific installation requirements, including a more complex set of steps. For example, AlignScore fact-checking, using Llama-2 requires two additional packages. For each feature or LLM example, check the readme file associated with it.

Using Docker#

The NeMo Guardrails toolkit can also be used through Docker. For details on how to build and use the Docker image see NeMo Guardrails with Docker.

What’s next?#