Skip to main content

Module run

Module run 

Source
Expand description

Drive a libsy algorithm to completion, making the model calls it offloads.

switchyard_libsy::Algorithm::run_stream is the whole libsy API: it yields a stream of steps and expects its consumer to serve every offloaded model call. run() is that consumer — it drives the stream with switchyard_libsy::drive, hands routing-time calls to a RoutedLlmClient, and serves the terminal routing outcome.

libsy owns the stream mechanics; what this module adds is per-target request preparation, ordered candidate fallback, and the libsy.client_call span around each candidate. Each completion candidate exhausts its backend retry budget before fallback advances. Routing calls stop on the first candidate’s failure unless the algorithm enables error recovery. A timeout stops completion calls.

A Responses continuation can refer to state through previous_response_id or conversation. ClientRouter sends native Responses state back to its provider; for Chat or Anthropic targets it retains the canonical history needed to materialize the continuation.

Structs§

ClientRouter
Resolves a routed call’s selected model to the client that serves it.

Functions§

decide
Run an algorithm to a routing decision without serving its terminal completion.
run
Run one request to completion, serving every offloaded model call with client.