CLI: export

Usage

vally export --eval-spec <path> [--eval-spec <path> ...] --exporter <specifier> [options] [-- key=value ...]

Description

Convert one or more eval specs to a format consumed by another evaluation tool. The export command validates each eval spec, resolves environments and stimuli, loads an exporter plugin, and invokes it to generate output files.

--eval-spec is repeatable. When a single spec is given, output is written directly to the output directory. When multiple specs are given, each spec’s output is written into its own subdirectory of the output directory, named after that spec’s (sanitized) name field.

Options

Flag	Type	Required	Description
--eval-spec <path>	string	Yes	Path to an eval.yaml file. Repeatable.
--exporter <specifier>	string	Yes	Exporter plugin to load (npm package name or local path)
--grader-plugin <specifier>	string	No	Grader plugin to load (npm package name or local path). Prevents unknown-grader-type errors for plugin graders during validation.
--output-dir <dir>	string	No	Directory for output files (default: ./<exporter-name>/)
--tag <key=values>	string	No	Filter stimuli by tag (repeatable, e.g., --tag cost=free,low)
--param <key=value>	string	No	Set a param value (repeatable, e.g. --param MODEL=gpt-4o). Overrides eval param files and .vally.yaml.
--verbose	boolean	No	Show resolved eval schema and plugin details

Exporter-specific options

Options after -- are forwarded to the exporter plugin as key=value pairs.

Exporter resolution

The --exporter specifier is resolved differently depending on its shape:

Specifier type	Examples	Resolved from
Local path	./my-exporter, ../plugins/exp, /abs/path	Current working directory
Bare package name	@contoso/vally-exporter-foo, my-exporter	Eval project directory (dirname(eval.yaml))

When multiple --eval-spec values are given, a bare exporter specifier is resolved from the first spec’s project directory, so a bare exporter is expected to be a shared dependency across the specs.

Exit codes

Code	Meaning
0	Export completed successfully
1	Validation error, plugin load failure, or export error

Examples

Basic export

vally export --eval-spec eval.yaml --exporter @contoso/vally-exporter-foo

With output directory and tag filter

vally export --eval-spec ./evals/my-eval/eval.yaml \
  --exporter @contoso/vally-exporter-foo \
  --output-dir ./exported \
  --tag cost=free,low

Pass exporter-specific options

vally export --eval-spec eval.yaml \
  --exporter @contoso/exporter \
  -- baseImage=ubuntu:22.04 format=compact

Bulk export (multiple specs)

vally export \
  --eval-spec ./evals/login/eval.yaml \
  --eval-spec ./evals/checkout/eval.yaml \
  --exporter @contoso/vally-exporter-foo \
  --output-dir ./exported

Each spec is written into its own subdirectory of ./exported, named after the spec’s name field (sanitized) — for example ./exported/login/ and ./exported/checkout/. If two specs resolve to the same subdirectory name (compared case-insensitively, for cross-platform safety), the command reports an error and exits without writing anything.