Skip to main content

drive

Function drive 

Source
pub async fn drive<F, Fut>(
    algorithm: Arc<dyn Algorithm>,
    request: Request,
    models: Arc<RuntimeModels>,
    serve: F,
) -> Result<RoutingOutcome>
where F: Fn(CallModel) -> Fut, Fut: Future<Output = Result<()>>,
Expand description

Drive Algorithm::run_stream to completion, handing each offloaded call to serve.

Returns the final RoutingOutcome. serve owns the call: it performs it however the host likes and must fulfill the promise with CallModel::respond. A failed model call belongs in respond — the algorithm may route around it. To stop routing on a model-call failure, return CallModel::fail instead. Returning Err from serve aborts the whole run. Calls are served concurrently, so an algorithm that offloads several at once (hedging, fan-out) gets real parallelism.

libsy performs no I/O; this is only the mechanics of consuming its own step stream, kept here so every host does not reimplement the same loop. switchyard-llm-client’s run is this function plus an HTTP client.