Vally
Vally
that makes your AI projects more reliable, more efficient, and more capable.
Run it as part of your dev workflow, your CI, or wherever evals need to happen.
How a run flows
Every evaluation moves through the same pipeline. A stimulus goes in, an executor runs your agent, the trajectory records every step, graders score what happened, and you get a score.
Hover or focus any stage for details
- The input that starts a run: a prompt, task, or message you want evaluated.
- Runs the thing under test. Point it at any agent, MCP server, or CLI. Your executor: Swap in the agent, MCP server, or CLI you want to test. Same pipeline contract.
- The full recorded trace of the run: every turn, tool call, and token spent.
- Checks that score the trajectory. Use the built-ins or write your own. Your graders: Write checks for exactly what your product needs. Same pipeline contract.
- The result: pass or fail, plus how well, how fast, and at what cost.
Measure how well, how fast, and why
Most eval tools tell you one thing: did it get the right answer? Vally tells you that too, and then it tells you what most tools miss. Because Vally captures the full trajectory, you can see past the final grade to
the tokens spent, the route taken, and the choices made
along the way. Then add graders that push your product to get there faster and cheaper.
Built to extend
Executors: bring your own agent
Point Vally at any MCP server, skillset, or CLI agent with a released executor. Need something they don’t cover? Write your own. The thing under test is yours, Vally runs it.
Graders: score what matters to you
Start with the built-in graders. Write your own when your product needs a check no one else has thought of, and push it toward the right answer, faster.
Environments: fast, or airtight
Run fast and local while you iterate. Switch to a fully isolated environment when you need a clean, repeatable result you can trust.
Reporters: results in your shape
Use the built-in dashboard and exports, or write a reporter that pipes results straight into the tools your team already watches.