Skip to main content

Algorithm

switchyard_libsy

Trait Algorithm 

Source 
pub trait Algorithm:
    Send
    + Sync
    + 'static {
    // Required methods
    fn name(&self) -> &str;
    fn route<'async_trait>(
        self: Arc<Self>,
        driver: Driver,
        request: Request,
    ) -> Pin<Box<dyn Future<Output = Result<Response>> + Send + 'async_trait>>
       where Self: 'async_trait;

    // Provided method
    fn run_stream(self: Arc<Self>, request: Request) -> StepStream { ... }
}
Expand description

An optimization strategy. Implement route; callers drive it with run_stream, serving each Step::CallModel it emits. switchyard-llm-client’s run is the ready-made consumer that does this over HTTP.

Methods take self: Arc<Self>: one algorithm (Arc<dyn Algorithm>) is shared across requests and run concurrently, so it owns its thread-safety and any shared state.

§Concurrency

A host may run the same algorithm concurrently for many requests. Implementations must synchronize their own mutable shared state. Each call to run_stream creates an independent Driver, so model-call promises and emitted Steps cannot cross between runs.

§Observability

run_stream creates a libsy.run span, and each offloaded model call creates a libsy.llm_call span. Decisions and failures are emitted through tracing; metrics use the global OpenTelemetry meter provider. The provider call itself belongs to the host, and is instrumented by whoever makes it.

Required Methods§

Source

fn name(&self) -> &str

Stable, low-cardinality name identifying this algorithm — the algorithm attribute on every span, metric, and log line the crate emits for its runs.

Source

fn route<'async_trait>( self: Arc<Self>, driver: Driver, request: Request, ) -> Pin<Box<dyn Future<Output = Result<Response>> + Send + 'async_trait>>
where Self: 'async_trait,

Run one request to completion: make model calls with Driver::call_model, publish Decisions with Driver::decide, and return the final Response. The method an algorithm implements; run_stream drives it.

Provided Methods§

Source

fn run_stream(self: Arc<Self>, request: Request) -> StepStream

Process a request to completion, returning a stream of Steps.

The consumer must fulfill every Step::CallModel before the algorithm can continue. Every run ends with exactly one terminal item — Step::Done on success, an Err item on failure, including when the algorithm panics. Dropping the stream aborts the spawned algorithm task.

Every invocation owns a separate Driver.

Implementors§