Reference

CLI Commandslint, eval, grade, compare, experiment, export, serve, ingest — flags, arguments, exit codes.

eval.yaml SchemaComplete specification for eval spec files.

Experiment File SchemaYAML format for `vally experiment run` (variants, matrix, vary, baseline).

SKILL.md FormatHow to write a valid skill definition file.

Grader CatalogAll built-in graders with config options and taxonomy.

Trajectory FormatThe JSON schema for captured agent behavior.

Scoring Functionspass@k, pass^k, and multi-trial metrics.