CLI: serve

Usage

vally serve [directory] [options]

Description

Start an HTTP server that provides a REST API and built-in web dashboard for exploring eval results.

There are three ways to use it:

# Serve from a directory (in-memory, no persistence)
vally serve ./results

# Serve from an existing database (no directory needed)
vally serve --store eval-history.db

# Ingest a directory into a database and serve
vally serve ./results --store eval-history.db

Arguments

Argument	Description
`[directory]`	Path to eval output directory. Optional when using `--store`

Options

Flag	Type	Default	Description
`--port <port>`	number	`3200`	Port to listen on
`--host <host>`	string	`127.0.0.1`	Host to bind to. Use `0.0.0.0` to expose externally.
`--cors`	boolean	`false`	Enable CORS for cross-origin browser requests
`--store <path>`	string	—	Persist data to a SQLite file (enables historical mode)

Exit codes

Code	Meaning
`0`	Server started successfully
`1`	Error (no runs found, bad port)

REST API

The server exposes a JSON API at /api/. Key endpoints:

Endpoint	Description
`GET /api/runs`	List all runs
`GET /api/runs/:id`	Run detail (stimuli, models, config)
`GET /api/runs/:id/outcomes`	Outcomes scoped to a run
`GET /api/runs/:id/matrix`	Score matrix (any metric)
`GET /api/runs/:id/ranking`	Model leaderboard
`GET /api/runs/:id/graders`	Grader failure analysis
`GET /api/runs/:id/tools`	Per-tool usage stats
`GET /api/outcomes`	List/filter outcomes across runs
`GET /api/outcomes/:id`	Full outcome with grader details
`GET /api/outcomes/:id/trajectory`	Raw trajectory events
`GET /api/compare?runs=id1,id2`	Cross-run comparison
`GET /api/metrics`	Self-describing metric definitions

All responses use standardized error envelopes ({ error: { code, message } }) and explicit pagination ({ page: { nextCursor, limit, hasMore, total } }).

Examples

# Basic usage — view results from an eval run
vally serve ./vally-results/

# Custom port
vally serve ./vally-results/ --port 8080

# Persistent database — keeps history across sessions
vally serve ./vally-results/ --store eval-history.db

# Query the API with curl
curl http://127.0.0.1:3200/api/runs
curl http://127.0.0.1:3200/api/runs/<run-id>/ranking

Security

Binds to 127.0.0.1 by default
CORS is disabled by default
Internal errors return a generic message (no paths or SQL fragments are exposed)