Struct HierarchicalRouter 

pub struct HierarchicalRouter { /* private fields */ }

Runs a stage router with a tier the judge picks.

Implementations

impl HierarchicalRouter

pub fn new( capable: ModelId, efficient: ModelId, config: HierarchicalRouterConfig, ) -> Result<Self>

Stacks the judge over a stage router across the same tier pair.

Errors on a configuration either algorithm rejects and on every_request.

A stage router carrying its own judge is allowed, but that judge sits ahead of the fall-open tier and so answers most of the turns this one set a tier for.

Trait Implementations

impl Algorithm for HierarchicalRouter

fn name(&self) -> &str

Stable, low-cardinality name identifying this algorithm — the algorithm attribute on every span, metric, and log line the crate emits for its runs.

fn route<'async_trait>( self: Arc<Self>, driver: Driver, request: Request, ) -> Pin<Box<dyn Future<Output = Result<RoutingOutcome>> + Send + 'async_trait>>

where Self: 'async_trait,

Run one request to completion: make routing-time model calls with Driver::call_model and return the terminal RoutingOutcome. The method an algorithm implements; run_stream drives it.

fn run_stream(self: Arc<Self>, request: Request) -> StepStream

Process a request to completion, returning a stream of Steps.