Struct CallLlmRequest 

pub struct CallLlmRequest { /* private fields */ }

The host-facing half of an offloaded model call, surfaced inside Step::CallLlm.

The host reads the routed request (get_routed) and the decision behind it (get_decision), performs (or delegates) the model call, and fulfills it with respond — unblocking the algorithm’s Driver::call_llm on the other side. switchyard-llm-client’s run is the ready-made consumer that does this for you.

Implementations

impl CallLlmRequest

pub fn get_routed(&self) -> &RoutedRequest

The routed request the host should serve; its decision.selected_model_id() names the model to hit.

pub fn get_request(&self) -> &Request

The model request to perform (the Request inside the routed request).

pub fn get_decision(&self) -> &Decision

The decision that led to this call; its selected_model_id() identifies the model to hit.

pub fn respond(self, result: Result<Response>) -> Result<()>

Fulfill the promise with the caller’s model-call result. Pass Err(..) to propagate a failed model call back to the algorithm. Consumes the promise: it can only be fulfilled once.