Skill Distillation

Skill distillation turns agent sessions into a reusable skill for the same kind of work. The model is not retrained. Switchyard saves the session history, uses it to update a SKILL.md, and makes the active skill available to later agent launches for the same namespace.

The current release adds the saved configuration and the Rust contracts that later implementations will share. It does not yet save session files, run distillation, update skills, import external runs, validate results, or mount skills into launched agents.

Configure It

Choose a short namespace for the task or workflow you want the skill to learn:

switchyard configure --skill-distillation tooluniverse-trialqa
switchyard configure --show

One namespace identifies one skill that improves over time. Keep using the same namespace when more sessions or trajectories should contribute to that skill; choose a different namespace when you want a separate skill. A namespace is not a session ID. In the automatic workflow, Switchyard will generate a separate internal session ID for each launcher run.

The namespace is stored in the user config and can be updated without provider credentials. To remove it:

switchyard configure --disable-skill-distillation

Intended Workflow

Once launcher support lands, the flow should be automatic:

configure a namespace
run an agent through switchyard launch
save the session under that namespace
distill when the session ends
create or update the namespace's active SKILL.md
load that active skill in the next launch for the same namespace

If no skill exists yet, the first distilled session creates one. Later sessions update the existing skill and keep enough history to inspect or roll back the change.

Skill distillation is namespace-based. The namespace is saved user configuration, not a per-request header. A future request may be recorded as part of a saved session, but a single HTTP request should not change which skill is being learned or used.

Session Ownership

The user configures only the namespace. In the automatic workflow, Switchyard will own session capture and the session lifecycle. Each switchyard launch will generate a separate session ID, record completed model turns as they pass through the local proxy, and finalize the project-local session record when the launched agent exits. The user will not provide a session ID, choose a storage path, or run a separate recorder.

The capture backend is an internal implementation detail. The initial implementation will use a Switchyard-managed project-local store; a later trajectory source or storage backend can replace or supplement it without changing the configuration or launcher workflow.

Config

Skill distillation config is stored in ~/.config/switchyard/config.json under the top-level skill_distillation key:

{
  "skill_distillation": {
    "namespace": "tooluniverse-trialqa"
  }
}

The config is intentionally small:

Field	Default	Meaning
namespace	unset	Stable name for one skill that improves over time. Many sessions or trajectories can contribute to it; it is not a session ID.

Namespaces must be safe local path components: letters, numbers, dot, underscore, and hyphen only. They cannot be . or ...

The top-level skill_distillation key is omitted when skill distillation is not configured. namespace is the only supported key today. Extra manually edited keys are rejected instead of being treated as inactive future options.

Decisions

Generated output should stay portable and easy to review. Switchyard should write SKILL.md and human-readable reports, not generated Python or other executable files.

Skill generation should happen after sessions finish, not during normal model request handling. Switchyard owns the session-to-skill lifecycle, including capture, distillation, validation, history, and launch-time skill loading. It should not turn every request into a memory update.

The automatic workflow still needs guardrails. Every skill update should record which sessions supported it, keep the previous active skill available for rollback, and produce review output. Later checks should flag answer leakage, source IDs, URLs, benchmark shortcuts, and task-specific details before a skill is trusted.

Planned Store Layout

Future work should use inspectable local files:

<project>/.switchyard/skill-distillation/<namespace>/
  sessions/<session-id>/
    session.json
    turns.jsonl
    stats.json
  distillation-ledger.jsonl
  candidates/<version-id>/
    skill/SKILL.md
    report.md
  active/SKILL.md
  history.jsonl

The ledger should track which saved sessions have already contributed to a skill update. That lets distillation use new sessions by default instead of depending on a long-lived lookback-count setting.

Rust Contracts

The switchyard-skill-distillation crate defines the source-neutral records and extension points shared by future implementations and adapters. It does not choose a provider, storage format, agent runtime, or model implementation.

The public contract includes:

• safe SkillNamespace, SkillEvidenceId, and SkillVersionId identifiers;
• versioned trajectories with task, execution, source, event, and outcome data;
• versioned skill candidates with required source-evidence provenance and optional validation reports; and
• async TrajectorySource, SkillDistiller, SkillValidator, and SkillStore traits.

The traits define workflow steps, but they do not run by themselves. No production Switchyard code uses them today. A Switchyard runtime must load saved trajectories and the active skill, build a distillation request, create and check a candidate, then save it and decide whether to make it active. This crate does not choose the code that implements each trait or decide when those steps run.

SkillEvidenceId is not a second session tracker. A session ID groups requests and turns while an agent is running. A SkillEvidenceId identifies the completed run after Switchyard saves it for distillation. The initial workflow will create one evidence record for each completed launcher session. Switchyard keeps this ID stable within the namespace. The Rust contract uses it to reject duplicate inputs in one distillation request and to record which evidence a skill candidate used. The code that creates an evidence record reuses a safe session ID or derives the evidence ID from it in a repeatable way. It can keep the original run or session ID in TrajectorySourceInfo.id.

Provider-specific event data stays inside JSON payload and metadata fields. The crate validates record invariants but does not redact source data, run a model, write files, or activate a candidate by itself. Adapters and runtime implementations own those operations.