Reference

CLI CommandsAll Vally CLI commands — flags, arguments, and exit codes.

eval.yaml SchemaComplete specification for eval spec files.

Experiment File SchemaYAML format for `vally experiment run` (variants, matrix, vary, baseline).

Grader CatalogAll built-in graders with config options and taxonomy.

Trajectory FormatThe JSON schema for captured agent behavior.

Scoring Functionspass@k, pass^k, and multi-trial metrics.

TelemetryWhat usage telemetry the CLI collects, what it never collects, and how to turn it off.