Schema Changes

Waza public artifacts use an explicit schemaVersion field so checked-in eval suites, baselines, and dashboard data remain stable across CLI upgrades.

Versioned artifacts

Artifact	Field	Current version
eval.yaml	schemaVersion	1.2
results.json	schemaVersion	1.2
snapshot.json	schemaVersion	1.0
Dashboard/SSE event envelope	schemaVersion	1.0

Policy

Schema versions use MAJOR.MINOR format with no patch component.

• MINOR changes are backward-compatible additions, usually optional fields. Readers accept same-major artifacts and warn when they see unknown fields.
• MAJOR changes are breaking. Readers refuse artifacts from a different major version and point to waza migrate <file>.
• Missing schemaVersion is interpreted as the current schema version (currently 1.2 for eval/result artifacts). Same-major minor differences are accepted and any unknown fields are warned about; cross-major mismatches are rejected.
• New eval/result artifacts should emit the current schemaVersion (currently 1.2). The version is automatically populated by the writer; you only need to set it manually when authoring fixtures or schema-pinned test data.

Migration command

Use waza migrate <file> when a reader reports an incompatible major version.

waza migrate eval.yaml
waza migrate results.json

For schema 1.0, the command is a no-op because there is no prior major version to migrate from.

Changelog

1.2

• Added optional EvalSpec.adversarial to declare built-in fault-injection packs and the on_unsafe_outcome policy consumed by waza adversarial --spec (issue #365).
• Added optional runs[].snapshot_path to results.json, pointing to the per-task snapshot.json artifact produced by waza run --snapshot (issue #367).
• Introduced snapshot.json as a separate task-snapshot artifact with its own independent schemaVersion: "1.0", prompt/fixture digests, environment allow-list capture, redaction metadata, and ordered tool-event replay tape (issue #367).

1.1

• Added optional checkpoints[] to task YAML for per-turn graders (after_turn, graders, on_failure). Backward-compatible: 1.0 task files load unchanged.
• Added optional checkpoints[] to results.json task results, recording per-turn grader outcomes.
• Added optional runs[].tool_events[] array to results.json (issue #366). Each entry captures one tool call with turn, sequence, tool_call_id, tool_name, args, result, success, error, and duration_ms. The legacy session_digest.tool_calls field is preserved.
• Bumped results.json schemaVersion default to 1.1. Readers accept 1.0 and 1.1 interchangeably (same major).

1.0

• Added schemaVersion to eval.yaml.
• Added schemaVersion to results.json.
• Added schemaVersion to the dashboard SSE event envelope type.
• Established same-major unknown-field warnings and cross-major reader errors.
• Added the waza migrate <file> command stub for future major migrations.