Trait StreamCodec

Source

pub trait StreamCodec: Send + Sync {
    // Required methods
    fn format(&self) -> FormatId;
    fn is_terminal_event(&self, event: &Value) -> bool;
    fn sse_done_is_terminal(&self) -> bool;
    fn stamp_streamed_response_model(
        &self,
        event: &mut Value,
        served_model: &str,
    );
    fn decode_event(
        &self,
        state: &mut StreamTranslationState,
        event: &Value,
    ) -> Vec<LlmResponseChunk>;
    fn encode_event(
        &self,
        state: &mut StreamTranslationState,
        event: LlmResponseChunk,
    ) -> Vec<Value>;
    fn finish(&self, state: &mut StreamTranslationState) -> Vec<Value>;

    // Provided method
    fn observe_replayed_event(
        &self,
        state: &mut StreamTranslationState,
        _raw: &Value,
        normalized: Vec<LlmResponseChunk>,
    ) { ... }
}

Expand description

Codec contract for one provider streaming event format.

Required Methods§

Source

fn format(&self) -> FormatId

Returns the stream format handled by this codec.

Source

fn is_terminal_event(&self, event: &Value) -> bool

Returns whether a JSON event establishes stream completion.

Source

fn sse_done_is_terminal(&self) -> bool

Returns whether the SSE [DONE] marker establishes stream completion.

The marker ends consumption even when this returns false; a terminal JSON event must then have been observed to avoid a premature-EOF error.

Source

fn stamp_streamed_response_model(&self, event: &mut Value, served_model: &str)

Overrides the model reported by an encoded or preserved event.

Called only when a served model was supplied, including for events emitted by finish.

Source

fn decode_event( &self, state: &mut StreamTranslationState, event: &Value, ) -> Vec<LlmResponseChunk>

Decodes one provider event into zero or more neutral events.

Source

fn encode_event( &self, state: &mut StreamTranslationState, event: LlmResponseChunk, ) -> Vec<Value>

Encodes one neutral event into zero or more provider events.

Source

fn finish(&self, state: &mut StreamTranslationState) -> Vec<Value>

Emits any terminal provider events needed after the source stream ends.

This is intentionally required on every codec. Some target formats need explicit terminal events after the source closes (for example, Anthropic message_delta/message_stop or Responses response.completed). Formats that have no source-close work should return an empty vector explicitly so the no-op behavior is a conscious codec-level choice.

Provided Methods§

Source

fn observe_replayed_event( &self, state: &mut StreamTranslationState, _raw: &Value, normalized: Vec<LlmResponseChunk>, )

Advances encoder state after an exact same-format event replay.

Exact replay returns the preserved provider JSON instead of the JSON emitted by Self::encode_event. The encoder must nevertheless observe the normalized chunks so Self::finish can close an incomplete stream without duplicating an already replayed terminal event. Codecs whose terminal state cannot be inferred from MessageStop alone may override this hook and inspect raw.