Skip to content

Decision Model-Related Protocols

A decision model evaluates a context against named questions and returns typed answers. An algorithm can use those answers to choose a model or decide what work to request next.

Switchyard represents this exchange with DecisionRequest and DecisionResponse. These are the types exchanged through Step::CallDecision.

System One is a common API for open decision models, including Jev. Switchyard supports it alongside OpenAI Decisions through the same protocol.

Question fields, answer types, and API mappings

  • Question reference: shared context, question IDs, Boolean/choice/score questions, and provider request mappings.
  • Answer reference: probabilities, rubric scores, confidence, usage, provider response mappings, and errors.

Describe the questions

One request carries shared context and independent questions keyed by ID. Each DecisionQuestion defines what to evaluate and which answer kind to return. The response uses the same question IDs, so the algorithm can match each answer to its question.

Here is a request asking whether a task contains private information. This is request construction only; it does not call a provider.

use std::collections::BTreeMap;
use serde_json::json;
use switchyard_protocol::{DecisionKind, DecisionQuestion, DecisionRequest};

let request = DecisionRequest {
    model: Some("privacy-judge".into()),
    context: json!({"task": "Summarize this customer's account history."}),
    questions: BTreeMap::from([(
        "contains_private_data".into(),
        DecisionQuestion {
            instructions: json!("Does the task contain private customer data?"),
            kind: DecisionKind::Boolean {
                true_description: Some(json!("Contains private customer data")),
                false_description: Some(json!("Contains only public information")),
            },
        },
    )]),
};

For required option IDs, rubric levels, and validation responsibilities, see question construction and validation.

Interpret the answers

DecisionResponse::answers maps the same question IDs to DecisionAnswer values. Each answer separates its typed value from optional provider_confidence. For the privacy question, ProbabilityTrue(Probability(0.8)) means an estimated 0.8 probability that the answer is true. It is not a routing choice until the algorithm applies its policy. Choice questions can name any supported set of candidates; their shape is not limited to two models.

The answer value reference explains rubric scores, distributions, and the difference between a Boolean judgment and an answer probability.

Understand conversion

For supported questions and answers, conversion is a direct, lossless mapping of decision data. IDs, answer probabilities, rubric positions, and confidence retain their meanings. Structured values become JSON text where OpenAI requires strings. This retains their content, rather than making the two wire formats identical or providing an arbitrary JSON round-trip guarantee.

Both adapters return probabilities for Boolean answers. A custom backend that returns only true or false can use BooleanEstimate::Value without inventing certainty. Optional distributions remain None when absent. The OpenAI adapter reports refusals as errors because DecisionValue has no refusal variant.

Own the conversion

When handling Step::CallDecision, use your existing client or a function to convert call.request into the backend's format, send it, and convert the answer into DecisionResponse. Pass that work to call.respond as a future returning libsy::Result<DecisionResponse>.

Match question IDs and kinds, and retain available distributions and confidence separately. The question reference links the System One and OpenAI adapter implementations. Decision conversions live in switchyard-llm-client; switchyard-translation handles LLM formats.

RoutedDecisionClient is an optional interface that lets Switchyard's clients and execution helpers work together. Direct run_stream handling does not require implementing it. See Mechanics of Libsy for the call-handling loop and cancellation behavior.

Use the supplied clients

You can instead use the public SystemOneClient or OpenAiDecisionClient from switchyard-llm-client. Use dependencies at the same API revision as libsy and the protocol crate.

This async excerpt sends the earlier request to System One. Set its model to an ID served by your endpoint. SYSTEM_ONE_ENDPOINT must include /v1/systemone. Running it makes a provider call and may incur a charge. The trait import makes the supplied client's call method available; you do not implement a client in this example.

use std::time::Duration;
use switchyard_llm_client::SystemOneClient;
use switchyard_protocol::RoutedDecisionClient;

let client = SystemOneClient::new(
    std::env::var("SYSTEM_ONE_ENDPOINT")?.parse()?,
    std::env::var("SYSTEM_ONE_API_KEY")?,
    Duration::from_secs(30),
)?;
let response = client.call(request).await?;

OpenAiDecisionClient::new takes the same arguments, with an endpoint ending in /v1/decisions and that provider's key. Each adapter makes one attempt per call; its timeout covers the request and reading the response body.

For the built-in execution path, register decision clients by model ID with ClientRouter::single_with_decision_clients or new_with_decision_clients, then use switchyard_llm_client::run. The algorithm still receives the same typed DecisionResponse whichever client performs the call.