Skip to content
Vally

Get Started

Vally is an extensible evaluation platform for AI agents — covering the full development loop: lint skill definitions, run agents against test cases, grade their behavior (not just their output), compare changes across runs, and track quality over time.

The path from zero to a running eval has three steps:

  1. Install the CLI — one npm install command, takes under a minute.
  2. Run your first eval — scaffold a spec, run an agent, read the scores.
  3. Add to CI — gate your pull requests automatically once you have a working eval.

If you’re new to the concepts behind Vally, How It Works explains the pipeline before you start.

Prerequisites: Node.js ≥ 22.12.0.

📦 Install VallyInstall the CLI globally with npm. Takes less than a minute.
🧪 Run Your First EvalScaffold a spec, run an agent, and grade the results end-to-end.
🚀 Add to CIIntegrate evaluation into your GitHub Actions pipeline.