Skip to main content

run

Function run 

Source
pub async fn run(
    algorithm: Arc<dyn Algorithm>,
    clients: ClientRouter,
    request: Request,
    models: Arc<RuntimeModels>,
    observer: Option<RunObserver>,
) -> Result<(ModelId, Response)>
Expand description

Run one request to completion, serving every offloaded model call with client.

Returns the model selected by the algorithm and the final Response. observer, when present, receives metadata for the routing outcome, each completed routing or answer call, and the routing overhead.

clients resolves each offloaded call to the client for the target the algorithm selected — an algorithm may route among targets served by different providers, so this is a per-call lookup, not one client for the whole run. Use ClientRouter::single when one client serves every target.

Routing calls are buffered; client failures stop the request unless the call enables recovery. Once routing completes, non-timeout failures may try the outcome’s ordered fallback candidates.