CLI: serve

Usage

vally serve [directory] [options]

Description

Start an HTTP server that provides a REST API and built-in web dashboard for exploring eval results.

There are three ways to use it:

# Serve from a directory (in-memory, no persistence)
vally serve ./results

# Serve from an existing database (no directory needed)
vally serve --store eval-history.db

# Ingest a directory into a database and serve
vally serve ./results --store eval-history.db

Arguments

Argument	Description
`[directory]`	Path to eval output directory. Optional when using `--store`

Options

Flag	Type	Default	Description
`--port <port>`	number	`3200`	Port to listen on
`--host <host>`	string	`127.0.0.1`	Host to bind to. Use `0.0.0.0` to expose externally.
`--cors`	boolean	`false`	Enable CORS for cross-origin browser requests
`--allowed-host <host>`	string	—	Additional `Host` header value to accept (repeatable). See Security.
`--store <path>`	string	—	Persist data to a SQLite file (enables historical mode)

Exit codes

Code	Meaning
`0`	Server started successfully
`1`	Error (no runs found, bad port)

REST API

The server exposes a JSON API at /api/. Key endpoints:

Endpoint	Description
`GET /api/runs`	List all runs
`GET /api/runs/:id`	Run detail (stimuli, models, config)
`GET /api/runs/:id/outcomes`	Outcomes scoped to a run
`GET /api/runs/:id/matrix`	Score matrix (any metric)
`GET /api/runs/:id/ranking`	Model leaderboard
`GET /api/runs/:id/graders`	Grader failure analysis
`GET /api/runs/:id/tools`	Per-tool usage stats
`GET /api/outcomes`	List/filter outcomes across runs
`GET /api/outcomes/:id`	Full outcome with grader details
`GET /api/outcomes/:id/trajectory`	Raw trajectory events
`GET /api/compare?runs=id1,id2`	Cross-run comparison
`GET /api/metrics`	Self-describing metric definitions

All responses use standardized error envelopes ({ error: { code, message } }) and explicit pagination ({ page: { nextCursor, limit, hasMore, total } }).

Examples

# Basic usage — view results from an eval run
vally serve ./vally-results/

# Custom port
vally serve ./vally-results/ --port 8080

# Persistent database — keeps history across sessions
vally serve ./vally-results/ --store eval-history.db

# Query the API with curl
curl http://127.0.0.1:3200/api/runs
curl http://127.0.0.1:3200/api/runs/<run-id>/ranking

Security

Binds to 127.0.0.1 by default
CORS is disabled by default
Internal errors return a generic message (no paths or SQL fragments are exposed)
Host-header validation (DNS-rebinding protection). The server rejects any request whose Host header is missing or not allowed with 403 forbidden_host. This blocks DNS-rebinding attacks, where a malicious web page repoints its hostname to 127.0.0.1 to bypass CORS and read your eval data. A Host is allowed when it is:
- a loopback name (localhost, 127.0.0.1, ::1),
- the value passed to --host,
- any IP literal (IPv4 or IPv6) — accessing the server directly by IP, including a LAN IP or a 0.0.0.0 bind, is inherently safe because rebinding requires a hostname the attacker can flip in DNS, or
- a name allowlisted via --allowed-host <host> (repeatable), e.g. when fronting the server with a custom domain.
Terminal window
```
# Allow a custom hostname (e.g. behind a reverse proxy)
vally serve ./results --allowed-host eval.internal.example.com
```
The /api/health endpoint is exempt so liveness probes can address the server by a Service DNS name; it returns no eval data.