CLI Commands

initScaffold a starter eval.yaml to get started quickly.

lintStatic checks on SKILL.md files and eval specs — no agent required.

evalRun stimuli against an agent and grade the results.

gradeRe-grade saved trajectories from stdin without re-running the agent.

Docker backendRun evals and standalone grading in isolated Docker containers.

oracleVerify graders against a golden patch — no agent run needed.

compareCompare a treatment run against a baseline with the LLM judge.

experimentRun multi-variant experiments and collect results for comparison.

simulateGenerate deterministic tool overrides from recorded trajectories.

exportExport eval results to different formats.

serveStart an analytics server and dashboard for eval results.

ingestImport eval results into a SQLite database for historical tracking.