CLI Commands

initScaffold a starter eval.yaml to get started quickly.

lintStatic checks on SKILL.md files and eval specs — no agent required.

evalRun stimuli against an agent and grade the results.

gradeRe-grade saved trajectories from stdin without re-running the agent.

oracleVerify graders against a golden patch — no agent run needed.

compareCompare a treatment run against a baseline with the LLM judge.

experimentRun multi-variant experiments and collect results for comparison.

exportExport eval results to different formats.

serveStart an analytics server and dashboard for eval results.

ingestImport eval results into a SQLite database for historical tracking.