Trait RoutedLlmClient 

pub trait RoutedLlmClient: Send + Sync {
    // Required method
    fn call<'life0, 'async_trait>(
        &'life0 self,
        ctx: Context,
        request: Request,
        decision: Arc<dyn Decision>,
    ) -> Pin<Box<dyn Future<Output = Result<Response, LlmClientError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}

Performs the actual model call for a target. This is the one piece of I/O the library does not own — a host implements it over its own transport (HTTP SDK, in-process model, mock). It serves a call the stream consumer chose not to override, reached as a routed request’s default_client.

Concurrency

A client may be shared by many targets and concurrent algorithm runs. Calls may overlap, so implementations must synchronize mutable state internally and should not serialize requests unless their transport requires it.

Required Methods

fn call<'life0, 'async_trait>( &'life0 self, ctx: Context, request: Request, decision: Arc<dyn Decision>, ) -> Pin<Box<dyn Future<Output = Result<Response, LlmClientError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Serve the call, returning the model’s response. Call the model named by decision.selected_model() — the target the algorithm routed to — mapping it to whatever provider model id this client hits. request.llm_request.model is the agent’s original name, carried through for reference, not a call target. ctx carries the request’s cross-cutting state.

Implementors