Testing scripts

It is possible to declare tests in the script function to validate the output of the script.

Declaring tests

The tests are added as an array of objects in the tests key of the script function.

proofreader.genai.mjs

scripts({
  ...,
  tests: {
    files: "src/rag/testcode.ts",
    rubrics: "is a report with a list of issues",
    facts: `The report says that the input string
      should be validated before use.`,
  }
})

Running tests

Visual Studio Code

• Open the Test Explorer view.
• Select your script in the tree and click the play icon button.

Command Line

Run this command from the workspace root.

npx genaiscript test proofreader

Known limitations

Currently, promptfoo treats the script source as the prompt text. Therefore, one cannot use assertions that also rely on the input text, such as answer_relevance.

• Read more about tests in the reference.

Next steps

Automate script execution using the command line interface (CLI).