Struct ModelInfraSignals 

pub struct ModelInfraSignals {
    pub workers: BTreeMap<String, WorkerLoad>,
    pub local_inflight: u64,
    pub local_spillovers: u64,
}

Backend worker gauges and requests still in flight through this host.

Fields

workers: BTreeMap<String, WorkerLoad>

One entry per independently scheduled worker, keyed by configured worker ID.

local_inflight: u64

All requests sent to this model that have not completed at this host.

local_spillovers: u64

Subset of local requests moved from the efficient tier.

Trait Implementations

impl Clone for ModelInfraSignals

fn clone(&self) -> ModelInfraSignals

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelInfraSignals

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ModelInfraSignals

fn default() -> ModelInfraSignals

Returns the “default value” for a type.