Vally 0.3.0
Released on 2026-04-29.
Features
- Add
--thresholdoption tovally evalfor overriding eval scoring thresholds during a command invocation. - Add optional
namefield to grader config for human-readable display names in reports. - Add
vally eval --keep-executor-session-logsto preserve native executorevents.jsonlfiles under--output-dirfor session visualizers such as agentviz. - Remote executor workspace sync contract —
finalizeWorkspace()hook onExecutorinterface enables remote executors to materialize workspace files locally before grading.workspaceStatusonTrajectoryenables fail-closed safety for file-based graders.requiresWorkspacemetadata onGraderBehaviorenables dynamic workspace-dependency detection. - Add
--known-domainsoption tovally lintfor URL reference scanning (#157) - Add
--allowed-external-depsoption tovally lintfor structural dependency scanning (#157) - Multi-model eval support (#140)
- Grader plugin system — ship graders in separate packages (#141)
- Grader improvements — regex matching, cross-platform normalization, better diagnostics (#137)
- Eval-authoring meta-eval and document directory support in
environment.files(#135) - Workspace caching for multi-trial runs (#132)
- JUnit XML reporter for eval pipeline (#128)
- Export command and exporter plugin system (#113)
Fixed
- Inject environment.skills into executor and honor
.vally.yamlpaths.skills(#139, #138) - Flatten multi-trial JUnit XML for CI parser compatibility (#131)
- Add
actions:readpermission for dorny/test-reporter (#130) - Fix bug when path to node contains spaces (#127)
Documentation
- Clarify GitHub Packages installation guide (#136)
- Document directory support in
environment.files(#135) - Docs for program, skill-invocation and tool-call graders (#124)
- Fix install instructions to be less destructive (#122)
- Document export command and architecture (#120)
- Common template for environment fields (vally-yaml and eval-spec) (#118)
- Document git and mcpServers in JSON schema (#116)