pub struct Driver { /* private fields */ }Expand description
How an algorithm’s route makes model calls.
Implementations§
Source§impl Driver
impl Driver
Sourcepub async fn call_model(
&self,
request: Request,
decision: Decision,
) -> Result<Response>
pub async fn call_model( &self, request: Request, decision: Decision, ) -> Result<Response>
Offload a model call: publish it as a Step::CallModel and await the consumer’s
Response. Errors if the stream is closed or the call failed.
The await is wrapped in a libsy.llm_call span measuring fulfillment as
the algorithm observes it (host queueing/serving included; a streamed
response resolves when its stream handle arrives); latency, outcome, and
token usage are recorded when it resolves. The provider call itself is the
host’s, and is instrumented by whoever makes it.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for Driver
impl RefUnwindSafe for Driver
impl Send for Driver
impl Sync for Driver
impl Unpin for Driver
impl UnsafeUnpin for Driver
impl UnwindSafe for Driver
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more