Skip to main content

Algorithm

switchyard_libsy

Trait Algorithm 

Source 
pub trait Algorithm:
    Send
    + Sync
    + 'static {
    // Required methods
    fn name(&self) -> &str;
    fn route<'async_trait>(
        self: Arc<Self>,
        driver: Driver,
        request: Request,
    ) -> Pin<Box<dyn Future<Output = Result<RoutingOutcome>> + Send + 'async_trait>>
       where Self: 'async_trait;

    // Provided method
    fn run_stream(self: Arc<Self>, request: Request) -> StepStream { ... }
}
Expand description

An optimization strategy. Implement route; callers drive it with run_stream, serving each Step::CallModel it emits. switchyard-llm-client’s run is the ready-made consumer that does this over HTTP.

Methods take self: Arc<Self>: one algorithm (Arc<dyn Algorithm>) is shared across requests and run concurrently, so it owns its thread-safety and any shared state.

§Concurrency

A host may run the same algorithm concurrently for many requests. Implementations must synchronize their own mutable shared state. Each call to run_stream creates an independent Driver, so model-call promises and emitted Steps cannot cross between runs.

§Observability

run_stream creates a libsy.run span, and each offloaded model call creates a nested libsy.llm_call span. Successful outcomes record their OutcomeMetadata::outcome_id on libsy.run, alongside selected_model_ids (an ordered OpenTelemetry string array). algorithm and switchyard.algorithm retain the run’s Algorithm::name. Optional evidence is the full JSON serialized as a string for consumers to parse. Keep it small and free of private data; nothing is filtered or redacted. These fields are span attributes, never metric labels.

The run/call observability helpers retain outcome status and operational metrics, but omit error details and arbitrary request extra metadata. Algorithms and hosts may emit their own logs. Errors still reach the caller unchanged. The host controls the tracing subscriber and global OpenTelemetry meter provider; libsy installs no exporter and performs no telemetry network I/O.

Required Methods§

Source

fn name(&self) -> &str

Stable, low-cardinality name identifying this algorithm — the algorithm attribute on every span, metric, and log line the crate emits for its runs.

Source

fn route<'async_trait>( self: Arc<Self>, driver: Driver, request: Request, ) -> Pin<Box<dyn Future<Output = Result<RoutingOutcome>> + Send + 'async_trait>>
where Self: 'async_trait,

Run one request to completion: make routing-time model calls with Driver::call_model and return the terminal RoutingOutcome. The method an algorithm implements; run_stream drives it.

Provided Methods§

Source

fn run_stream(self: Arc<Self>, request: Request) -> StepStream

Process a request to completion, returning a stream of Steps.

The consumer must fulfill every Step::CallModel before the algorithm can continue. Every run ends with exactly one terminal item — Step::Done on success, an Err item on failure, including when the algorithm panics. Dropping the stream aborts the spawned algorithm task.

Every invocation owns a separate Driver.

Implementors§