Struct ClientRouter 

pub struct ClientRouter { /* private fields */ }

Resolves a routed call’s selected model to the client that serves it.

An algorithm routes among named targets; which provider each target lives on is the host’s concern, and two targets in one run may sit on different providers. A router owns that mapping and resolves the client for each kind of call.

Cloning is cheap — the mapping is shared, so one router can serve every request.

Implementations

impl ClientRouter

pub fn new(by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>) -> Self

Build a router over model name -> client, for targets spread across providers.

pub fn new_with_target_prompts( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, ) -> Self

Build a router with target prompts used by run and decide.

routing_answer_target identifies a routing-time call whose response may become the answer. Pass None when routing only selects a later completion target. Prompt keys and routing_answer_target use the resolved model IDs stored in by_model. This constructor treats every model in by_model as a possible completion target. Use Self::new_with_completion_targets to exclude classifier-only models from the comparison of completion clients.

pub fn new_with_completion_targets( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, completion_targets: &[ModelId], ) -> Self

Build a router with an explicit list of models that can answer requests.

by_model contains callable LLMs, including LLM classifiers. Only the model IDs in completion_targets determine whether native Responses state needs a routing pin. Cross-format state is recorded lazily when a Responses request uses a Chat or Anthropic target. target_prompts and routing_answer_target have the same meaning as in Self::new_with_target_prompts.

pub fn new_with_decision_clients( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, decision_clients: HashMap<ModelId, Arc<dyn RoutedDecisionClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, completion_targets: &[ModelId], ) -> Self

Build a router with separate LLM and decision mappings, shared across clones. LLM prompt and completion settings follow Self::new_with_completion_targets.

pub fn single(client: Arc<dyn RoutedLlmClient>) -> Self

A router that serves every model with one client — the single-provider case.

TranslatingLlmClient already maps model names to backends internally and rejects ones it does not know, so enumerating them here would only duplicate that.

pub fn single_with_decision_clients( client: Arc<dyn RoutedLlmClient>, decision_clients: HashMap<ModelId, Arc<dyn RoutedDecisionClient>>, ) -> Self

Use one client for all LLM targets and explicit clients for decision targets.

pub fn without_rate_limit_fallback(self, models: HashSet<ModelId>) -> Self

Stop completion fallback on HTTP 429 for the given models.

For translating clients, also use TranslatingLlmClient::without_rate_limit_cooldown so later calls are not skipped with a generic cooldown error.

pub fn route( &self, model: &ModelId, ) -> Result<&Arc<dyn RoutedLlmClient>, LlmClientError>

The client that serves model.

Errors with LlmClientError::Configuration when the router maps models and has no entry for this one, rather than silently sending the call to another provider.

Trait Implementations

impl Clone for ClientRouter

fn clone(&self) -> ClientRouter

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl FromIterator<(ModelId, Arc<dyn RoutedLlmClient>)> for ClientRouter

fn from_iter<I: IntoIterator<Item = (ModelId, Arc<dyn RoutedLlmClient>)>>( iter: I, ) -> Self

Creates a value from an iterator.