nemoguardrails.types.LLMModel

class nemoguardrails.types.LLMModel

Bases: Protocol

Protocol that all LLM backends must implement.

Adapters wrap provider-specific SDKs (LangChain, LiteLLM, OpenAI, etc.) behind this interface so the core pipeline remains framework-agnostic.

prompt accepts either a plain string or a list of ChatMessage objects. Adapters convert ChatMessage to whatever their SDK expects. **kwargs are forwarded to the underlying SDK (e.g. temperature, max_tokens).

async generate_async(prompt, *, stop=None, **kwargs)

Return type:

LLMResponse

Parameters:

• prompt (str | List[ChatMessage])

• stop (List[str] | None)

stream_async(prompt, *, stop=None, **kwargs)

Implementations must be async generator functions (use yield).

Return type:

AsyncIterator[LLMResponseChunk]

Parameters:

• prompt (str | List[ChatMessage])

• stop (List[str] | None)

property model_name: str

property provider_name: str | None

property provider_url: str | None

__init__(*args, **kwargs)