Streaming Configuration#

NeMo Guardrails supports streaming out of the box when using the stream_async() method. No configuration is required to enable basic streaming.

When you have output rails configured, you need to explicitly enable streaming for them to process tokens in chunked mode.

Quick Example#

When using streaming with output rails:

rails:
  output:
    flows:
      - self check output
    streaming:
      enabled: True
      chunk_size: 200
      context_size: 50

The following guides provide detailed documentation for streaming configuration.

Output Rail Streaming

Configure how output rails process streamed tokens in chunked mode.