Skip to content

Commands

A full list of the CLI command and its respective help text.

run

Usage: genaiscript run [options] <script> [files...]
Runs a GenAIScript against files.
Options:
-m, --model <string> 'large' model alias (default)
-sm, --small-model <string> 'small' alias model
-vm, --vision-model <string> 'vision' alias model
-lp, --logprobs enable reporting token probabilities
-tlp, --top-logprobs <number> number of top logprobs (1 to 5)
-ef, --excluded-files <string...> excluded files
-egi, --exclude-git-ignore exclude files that are ignored through the .gitignore file in the workspace root
-ft, --fallback-tools Enable prompt-based tools instead of builtin LLM tool calling builtin tool calls
-o, --out <string> output folder. Extra markdown fields for output and trace will also be generated
-rmo, --remove-out remove output folder if it exists
-ot, --out-trace <string> output file for trace
-od, --out-data <string> output file for data (.jsonl/ndjson will be aggregated). JSON schema information and validation will be included if available.
-oa, --out-annotations <string> output file for annotations (.csv will be rendered as csv, .jsonl/ndjson will be aggregated)
-ocl, --out-changelog <string> output file for changelogs
-pr, --pull-request <number> pull request identifier
-prc, --pull-request-comment [string] create comment on a pull request with a unique id (defaults to script id)
-prd, --pull-request-description [string] create comment on a pull request description with a unique id (defaults to script id)
-prr, --pull-request-reviews create pull request reviews from annotations
-j, --json emit full JSON response to output
-y, --yaml emit full YAML response to output
-fe, --fail-on-errors fails on detected annotation error
-r, --retry <number> number of retries (default: "10")
-rd, --retry-delay <number> minimum delay between retries (default: "1000")
-md, --max-delay <number> maximum delay between retries (default: "10000")
-l, --label <string> label for the run
-t, --temperature <number> temperature for the run
-tp, --top-p <number> top-p for the run
-mt, --max-tokens <number> maximum completion tokens for the run
-mdr, --max-data-repairs <number> maximum data repairs
-mtc, --max-tool-calls <number> maximum tool calls for the run
-se, --seed <number> seed for the run
-em, --embeddings-model <string> embeddings model for the run
--cache enable LLM result cache
-cn, --cache-name <name> custom cache file name
-cs, --csv-separator <string> csv separator (default: "\t")
-ae, --apply-edits apply file edits
--vars <namevalue...> variables, as name=value, stored in env.vars. Use environment variables GENAISCRIPT_VAR_name=value to pass variable through the environment
-rr, --run-retry <number> number of retries for the entire run
-h, --help display help for command

test

Usage: genaiscript test [options] [command]
Options:
-h, --help display help for command
Commands:
run [options] [script...] Runs the tests for scripts
list [options] List available tests in workspace
view Launch test viewer
help [command] display help for command

test run

Usage: genaiscript test run [options] [script...]
Runs the tests for scripts
Arguments:
script Script ids. If not provided, all scripts
are tested
Options:
-m, --model <string> model for the run
-sm, --small-model <string> small model for the run
-vm, --vision-model <string> 'vision' alias model
--models <models...> models to test where mode is the key
value pair list of m (model), s (small
model), t (temperature), p (top-p)
-o, --out <folder> output folder
-rmo, --remove-out remove output folder if it exists
--cli <string> override path to the cli
-td, --test-delay <string> delay between tests in seconds
--cache enable LLM result cache
-v, --verbose verbose output
-pv, --promptfoo-version [version] promptfoo version, default is 0.97.0
-os, --out-summary <file> append output summary in file
--groups <groups...> groups to include or exclude. Use :!
prefix to exclude
-h, --help display help for command

test list

Usage: genaiscript test list [options]
List available tests in workspace
Options:
--groups <groups...> groups to include or exclude. Use :! prefix to exclude
-h, --help display help for command

test view

Usage: genaiscript test view [options]
Launch test viewer
Options:
-h, --help display help for command

scripts

Usage: genaiscript scripts|script [options] [command]
Utility tasks for scripts
Options:
-h, --help display help for command
Commands:
list List all available scripts in workspace
create <name> Create a new script
fix fix all definition files
compile [folders...] Compile all scripts in workspace
model [options] [script] List model connection information for scripts
help [command] display help for command

scripts list

Usage: genaiscript scripts list [options]
List all available scripts in workspace
Options:
-h, --help display help for command

scripts create

Usage: genaiscript scripts create [options] <name>
Create a new script
Arguments:
name Name of the script
Options:
-h, --help display help for command

scripts fix

Usage: genaiscript scripts fix [options]
fix all definition files
Options:
-h, --help display help for command

scripts compile

Usage: genaiscript scripts compile [options] [folders...]
Compile all scripts in workspace
Arguments:
folders Pattern to match files
Options:
-h, --help display help for command

scripts model

Usage: genaiscript scripts model [options] [script]
List model connection information for scripts
Arguments:
script Script id or file
Options:
-t, --token show token
-h, --help display help for command

cache

Usage: genaiscript cache [options] [command]
Cache management
Options:
-h, --help display help for command
Commands:
clear [name] Clear cache
help [command] display help for command

cache clear

Usage: genaiscript cache clear [options] [name]
Clear cache
Arguments:
name Name of the cache, tests
Options:
-h, --help display help for command

retrieval

Usage: genaiscript retrieval|retreival [options] [command]
RAG support
Options:
-h, --help display help for command
Commands:
search [options] <query> [files...] Search using vector embeddings
similarity
fuzz [options] <query> [files...] Search using string distance
help [command] display help for command
Usage: genaiscript retrieval search [options] <query> [files...]
Search using vector embeddings similarity
Options:
-ef, --excluded-files <string...> excluded files
-tk, --top-k <number> maximum number of results
-h, --help display help for command

retrieval fuzz

Usage: genaiscript retrieval fuzz [options] <query> [files...]
Search using string distance
Options:
-ef, --excluded-files <string...> excluded files
-tk, --top-k <number> maximum number of results
-h, --help display help for command

serve

Usage: genaiscript serve [options]
Start a GenAIScript local server
Options:
-p, --port <number> Specify the port number, default: 8003
-k, --api-key <string> API key to authenticate requests
-h, --help display help for command

parse

Usage: genaiscript parse|parsers [options] [command] <file...>
Parse various outputs
Arguments:
file input JSONL files
Options:
-h, --help display help for command
Commands:
data [options] <file> Convert CSV, YAML, TOML, INI, XLSX, XML, MD/X
frontmatter or JSON data files into various
formats
fence <language> <file> Extracts a code fenced regions of the given type
pdf [options] <file> Parse a PDF into text and images
docx <file> Parse a DOCX into texts
html-to-text <file> Parse an HTML file into text
code <file> [query] Parse code using tree sitter and executes a
query
tokens [options] <files...> Count tokens in a set of files
jsonl2json Converts JSONL files to a JSON file
prompty [options] <file...> Converts .prompty files to genaiscript
jinja2 [options] <file> Renders Jinj2 or prompty template

parse data

Usage: genaiscript parse data [options] <file>
Convert CSV, YAML, TOML, INI, XLSX, XML, MD/X frontmatter or JSON data files
into various formats
Options:
-f, --format <string> output format (choices: "json", "json5", "yaml",
"ini", "csv", "md")
-h, --help display help for command

parse fence

Usage: genaiscript parse fence [options] <language> <file>
Extracts a code fenced regions of the given type
Options:
-h, --help display help for command

parse pdf

Usage: genaiscript parse pdf [options] <file>
Parse a PDF into text and images
Options:
-i, --images extract images
-o, --out <string> output folder
-h, --help display help for command

parse docx

Usage: genaiscript parse docx [options] <file>
Parse a DOCX into texts
Options:
-h, --help display help for command

parse html-to-text

Usage: genaiscript parse html-to-text [options] <file>
Parse an HTML file into text
Options:
-h, --help display help for command

parse code

Usage: genaiscript parse code [options] <file> [query]
Parse code using tree sitter and executes a query
Options:
-h, --help display help for command

parse tokens

Usage: genaiscript parse tokens [options] <files...>
Count tokens in a set of files
Options:
-ef, --excluded-files <string...> excluded files
-h, --help display help for command

parse jsonl2json

Usage: genaiscript parse jsonl2json [options]
Converts JSONL files to a JSON file
Options:
-h, --help display help for command

parse prompty

Usage: genaiscript parse prompty [options] <file...>
Converts .prompty files to genaiscript
Arguments:
file input JSONL files
Options:
-o, --out <string> output folder
-h, --help display help for command

parse jinja2

Usage: genaiscript parse jinja2 [options] <file>
Renders Jinj2 or prompty template
Arguments:
file input Jinja2 or prompty template file
Options:
--vars <namevalue...> variables, as name=value passed to the template
-h, --help display help for command

info

Usage: genaiscript info [options] [command]
Utility tasks
Options:
-h, --help display help for command
Commands:
help Show help for all commands
system Show system information
env [options] [provider] Show .env information

info help

Usage: genaiscript info help [options]
Show help for all commands
Options:
-h, --help display help for command

info system

Usage: genaiscript info system [options]
Show system information
Options:
-h, --help display help for command

info env

Usage: genaiscript info env [options] [provider]
Show .env information
Options:
-t, --token show token
-e, --error show errors
-h, --help display help for command