Vally 0.12.0

Released on 2026-07-30.

vally eval no longer statically validates executor names: the unknown-executor lint check was removed, and unknown executors now fail at runtime with an “available executors” error. (#812)
environment.git blocks are now validated at load time for every environment (including named environments not referenced by an eval), so specs with unknown or invalid nested git fields that previously loaded silently now fail. (#811)

New OpenTelemetry CLI usage telemetry emits one PII-safe span per command; it is on by default and can be disabled via VALLY_TELEMETRY_OPTOUT or DO_NOT_TRACK. (#755)
New --executor <name> flag on vally eval overrides defaults.executor for every eval in the run. (#812)
The diff-contains, diff-not-contains, and diff-empty graders now support 0-based turn: scoping for multi-turn stimuli. (#814)
The max-repeat and loop-outcome graders gain opt-in periodic loop detection via max_cycle_period (default 1, up to 10). (#771)

vally serve now waits for a genuine listening event, so bind failures such as EADDRINUSE surface as a clean error instead of crashing after a misleading “listening” log. (#802)
A Claude CLI hard timeout is no longer misreported as a launch failure, preserving the captured partial output. (#780)
File graders now match dotfiles (e.g. .gitignore, files under .github/) and report an indeterminate failure on unreadable candidates instead of a false pass. (#796)
--verbose again shows the Agent Output section for failing trials in multi-trial runs. (#775)

The eval console summary now explains execution failures (e.g. failed: N/M trial(s) errored during execution) instead of showing a red ✘ beside a passing score. (#770)
CLI help output is restyled with bold uppercase section headers and a LEARN MORE footer linking to the docs. (#794)