pub struct Driver { /* private fields */ }Expand description
The offload channel handed to an algorithm’s
create_run_task. The algorithm makes model calls
with call_llm_target (or call_llm) and
publishes its Decisions with info; each call is offloaded to the
request’s Step stream and awaits the consumer’s response. The step channel is
bounded, so the consumer paces the algorithm one step at a time.
Implementations§
Source§impl Driver
impl Driver
Sourcepub fn record_counter(
&self,
name: &'static str,
delta: u64,
attributes: impl IntoIterator<Item = MetricAttribute>,
)
pub fn record_counter( &self, name: &'static str, delta: u64, attributes: impl IntoIterator<Item = MetricAttribute>, )
Add a delta to an algorithm-defined OpenTelemetry counter and report the same event to this run’s observer. Names and attributes must be stable and low-cardinality; never include request or session data.
Sourcepub fn record_histogram(
&self,
name: &'static str,
sample: f64,
attributes: impl IntoIterator<Item = MetricAttribute>,
)
pub fn record_histogram( &self, name: &'static str, sample: f64, attributes: impl IntoIterator<Item = MetricAttribute>, )
Record one sample in an algorithm-defined OpenTelemetry histogram and report the same event to this run’s observer. Non-finite samples are dropped.
Sourcepub async fn call_llm(&self, routed: RoutedRequest) -> Result<Response>
pub async fn call_llm(&self, routed: RoutedRequest) -> Result<Response>
Offload a model call: publish routed as a Step::CallLlm and await the
consumer’s Response. The call’s context travels inside
routed.ctx. Errors if the stream is closed or the call failed.
The await is wrapped in a libsy.llm_call span measuring fulfillment as
the algorithm observes it (host queueing/serving included; a streamed
response resolves when its stream handle arrives); latency, outcome, and
token usage are recorded when it resolves. The provider call itself gets a
libsy.client_call span when Algorithm::run serves it.
Sourcepub async fn call_llm_target(
&self,
ctx: Context,
target: &LlmTarget,
request: Request,
decision: Arc<dyn Decision>,
) -> Result<Response>
pub async fn call_llm_target( &self, ctx: Context, target: &LlmTarget, request: Request, decision: Arc<dyn Decision>, ) -> Result<Response>
Offload a call to target: pair request with decision and the target’s
default client into a RoutedRequest, then publish it (see
call_llm). The convenience most algorithms use;
decision.selected_model() names the model to hit, and request’s
model is left untouched.