Vally 0.9.0

Released on 2026-07-13.

LLM-as-judge graders (prompt, panel) now accept a configurable reasoning_effort (low/medium/high/xhigh), settable via defaults.judge_reasoning_effort, --judge-reasoning-effort, the EVAL_JUDGE_REASONING_EFFORT env var, or per-grader/per-model config. (#660)
New environment.commandTimeout field to override the 60s default timeout for environment setup commands. (#620)
The claude-cli executor now supports multi-turn stimuli. (#665)
vally experiment run gains a --verbose flag, matching vally eval. (#666)
tool-calls grader gains an optional pattern — a tool-agnostic regex matched against a call’s full serialized arguments, for content checks across a heterogeneous tool set. (#686)
environment.git supports a new type: clone mode to materialize the workspace from a remote repo (url), with optional ref, shallow, and sparse options. (#691)
New loop-outcome grader — deterministically asserts whether an agent got stuck in a loop (via excess retries or hitting the simulation iteration cap), matching an expected should_loop outcome. (#683)

Eval report footnotes now use standard GFM syntax so they render correctly outside github.com. (#687)
Hardened temp-dir cleanup against transient Windows EBUSY locks. (#692)
Eval runs are now hermetic by default: each run pins COPILOT_HOME to a fresh, empty per-run directory so the agent no longer reads the developer’s real ~/.copilot config. Set EVALUATE_USE_HOST_COPILOT_HOME=1 to opt out. (#656)