Skip to main content

Algorithm

switchyard_libsy

Trait Algorithm 

Source 
pub trait Algorithm:
    Send
    + Sync
    + 'static {
    // Required methods
    fn name(&self) -> &str;
    fn create_run_task<'async_trait>(
        self: Arc<Self>,
        ctx: Context,
        driver: Driver,
        request: Request,
    ) -> Pin<Box<dyn Future<Output = Result<Response>> + Send + 'async_trait>>
       where Self: 'async_trait;

    // Provided methods
    fn process_signals<'async_trait>(
        self: Arc<Self>,
        signals: Signals,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait { ... }
    fn count_tokens_client(&self) -> Option<Arc<dyn RoutedLlmClient>> { ... }
    fn count_tokens<'life0, 'async_trait>(
        &'life0 self,
        request: Request,
    ) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
    fn run_stream(
        self: Arc<Self>,
        ctx: Context,
        request: Request,
        observer: Option<RunObserver>,
    ) -> StepStream { ... }
    fn run<'async_trait>(
        self: Arc<Self>,
        ctx: Context,
        request: Request,
    ) -> Pin<Box<dyn Future<Output = Result<(Vec<Arc<dyn Decision>>, Response)>> + Send + 'async_trait>>
       where Self: 'async_trait { ... }
    fn run_observed<'async_trait>(
        self: Arc<Self>,
        ctx: Context,
        request: Request,
        observer: Option<RunObserver>,
    ) -> Pin<Box<dyn Future<Output = Result<(Vec<Arc<dyn Decision>>, Response)>> + Send + 'async_trait>>
       where Self: 'async_trait { ... }
}
Expand description

An optimization strategy. Implement create_run_task; callers drive it with the provided run (serve calls, get the answer) or run_stream (drive the Step stream yourself).

Methods take self: Arc<Self>: one algorithm (Arc<dyn Algorithm>) is shared across requests and run concurrently, so it owns its thread-safety and any shared state.

§Concurrency

A host may run the same algorithm concurrently for many requests. Implementations must synchronize their own mutable shared state. Each call to run_stream creates an independent Driver, so model-call promises and emitted Steps cannot cross between runs.

§Observability

run_stream creates a libsy.run span, and each offloaded model call creates a libsy.llm_call span. run additionally wraps calls it serves through a target’s default client in libsy.client_call. Decisions and failures are emitted through tracing; metrics use the global OpenTelemetry meter provider.

Required Methods§

Source

fn name(&self) -> &str

Stable, low-cardinality name identifying this algorithm — the algorithm attribute on every span, metric, and log line the crate emits for its runs.

Source

fn create_run_task<'async_trait>( self: Arc<Self>, ctx: Context, driver: Driver, request: Request, ) -> Pin<Box<dyn Future<Output = Result<Response>> + Send + 'async_trait>>
where Self: 'async_trait,

Run one request to completion: make model calls with Driver::call_llm_target, publish Decisions with Driver::info, and return the final Response. The method an algorithm implements; run / run_stream drive it. ctx carries the request’s cross-cutting values (today: the algorithm’s telemetry label in Context::values).

Provided Methods§

Source

fn process_signals<'async_trait>( self: Arc<Self>, signals: Signals, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait,

Feed the algorithm agentic-stack events (tool results, budgets, etc.). The reference algorithms ignore signals; a stateful algorithm updates its own (interior-mutable) state. Takes self: Arc<Self> like the other run methods.

Source

fn count_tokens_client(&self) -> Option<Arc<dyn RoutedLlmClient>>

The client count_tokens forwards to: the first of this algorithm’s targets whose client can count tokens (an Anthropic upstream). The default is None — an algorithm with no Anthropic target does not support token counting.

CAVEAT: this picks the first Anthropic target, not a routed one — count_tokens is a direct passthrough, so it does not run the routing cascade. For a route with several Anthropic tiers “first” is arbitrary; choosing which tier count_tokens should reflect is deferred.

Source

fn count_tokens<'life0, 'async_trait>( &'life0 self, request: Request, ) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Count the tokens request would use — a direct passthrough to this algorithm’s Anthropic target (via count_tokens_client), not a routed call. Token counting is a pre-flight estimate with no routing decision, so it deliberately bypasses the classifier cascade (which runs only for completions via run). Returns the upstream’s JSON verbatim. Errors when the algorithm has no Anthropic target.

Source

fn run_stream( self: Arc<Self>, ctx: Context, request: Request, observer: Option<RunObserver>, ) -> StepStream

Process a request to completion, returning a stream of Steps.

The consumer must fulfill every Step::CallLlm before the algorithm can continue. The bounded step channel applies backpressure when the consumer is not polling. A successful run ends with Step::ReturnToAgent; a failure is emitted as an Err item. Dropping the stream aborts the spawned algorithm task.

Every invocation owns a separate Driver. observer, when present, receives each completed model call and, after a successful routed run, its routing overhead.

Source

fn run<'async_trait>( self: Arc<Self>, ctx: Context, request: Request, ) -> Pin<Box<dyn Future<Output = Result<(Vec<Arc<dyn Decision>>, Response)>> + Send + 'async_trait>>
where Self: 'async_trait,

Process a request to completion, returning the final Response and the trace of Decisions the algorithm made along the way.

Source

fn run_observed<'async_trait>( self: Arc<Self>, ctx: Context, request: Request, observer: Option<RunObserver>, ) -> Pin<Box<dyn Future<Output = Result<(Vec<Arc<dyn Decision>>, Response)>> + Send + 'async_trait>>
where Self: 'async_trait,

Process a request to completion while reporting each model call to observer.

Implementors§