Vally 0.17.0

Released on 2026-09-23.

Unknown stimulus fields in eval.yaml are now rejected at runtime instead of silently ignored. (#1176)
LLM judges always read evidence from temporary files; output_delivery: inline is rejected and output_delivery: workspace is deprecated. (#1188)
Public API (@microsoft/vally): executor OTel “telemetry” APIs are renamed to “tracing” (e.g. configureTelemetry() → configureTracing(), ExecutorTelemetryConfig → TracingConfig), and backends can now receive tracing config too. (#1210)
The default LLM judge model is now gpt-5.6-terra. (#1184)

vally oracle accepts a golden_command input that runs a known-good workflow against the workspace before grading. (#1113)
stimulus.simulation can simulate MCP tool responses. (#1174)
.vally.yaml supports root-level and per-suite metadata, exposed to reporters via the run context. (#1177, #1187)
vally grade gains --backend and --backend-args for offline grading through a selected backend. (#1220)
The CLI shows a status indicator while reporter plugins run. (#1212)
ATIF conversion gains opt-in toolResultStatus: "require-explicit" and subagentIdentity: "instance" options. (#1225)

ATIF conversion honors producer-recorded tool failures instead of marking every tool result successful. (#1225)
Windows Harbor image builds retry transient VC++ runtime downloads and provision runtimes through a packaged script. (#1159, #1182)