Skip to content
Vally

Release notes

Version 0.21.0

Released

Latest release

Features

  • Docker evals can select different task and grading images per eval, and resources can limit CPU, memory, and PIDs for task and grading containers.
  • Eval specs can provide defaults for explicitly selected exporters.
  • Harbor uses an OS-specific base image when none is configured.
  • LLM judges can configure evidence snapshot limits and read budgets while retaining the existing defaults.
  • Copilot model summaries show resolved model selections without changing model identity or grouping.
  • Tool trajectory events include end timestamps for accurate per-step durations, with a serialized event type and converter for reporter integrations.
  • The Copilot SDK executor natively dispatches explicit /skill-name prompts, including skills that cannot be invoked by the model.
  • Oracle baselines support per-grader oracle_baseline: allow-pass to report legitimate passes as N/A under –no-golden-input, including for LLM and plugin graders.
  • Harbor task metadata preserves resolved eval and stimulus tags.

Fixed

  • Invalid grading results are reported as ERROR rather than FAIL, while valid failing verdicts remain FAIL.
  • Named environments resolve from each eval’s project configuration.
  • Turn-scoped graders work with joined Harbor exports when the agent reconstructs the original conversation.
  • TypeScript plugins load correctly.
  • Core package builds work with npm 12 and Git tarballs on Windows.