Struct ServingObservation 

pub struct ServingObservation {
    pub received_at: Instant,
    pub effective_prefill_tokens: usize,
    pub active_prefill_tokens: Option<usize>,
}

Serving observations for one candidate model. Never populated from caller headers.

Fields

received_at: Instant

Local receipt time; avoids relying on clocks synchronized with the serving system.

effective_prefill_tokens: usize

Work remaining after the serving system applies cache credits.

active_prefill_tokens: Option<usize>

Current tracked prefill work, when available.

Trait Implementations

impl Clone for ServingObservation

fn clone(&self) -> ServingObservation

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServingObservation

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.