Installation

This is the first of three steps to run your first eval.

Prerequisites

• Install Node.js >=22.14.0, which includes npm.
• Make sure your GitHub account has an active Copilot subscription for the default runtime used later.

Install the CLI

Install the Vally CLI with npm:

npm install -g @microsoft/vally-cli

Verify installation

vally --version

You should see the version number printed.

Tip

Next: Set up the default runtime so Vally can execute an agent.

Optional: Use Vally as a library

If you want to use Vally programmatically (custom graders, pipeline integration), install the core package:

npm install @microsoft/vally

Then import what you need from the relevant subpath:

import type { Grader, GraderInput, GraderResult } from "@microsoft/vally/graders";
import { createGraderRegistry } from "@microsoft/vally";

See the Writing Custom Graders guide for a full walkthrough.

Tip

The CLI is a thin wrapper around the core library. Anything the CLI can do, you can do programmatically with more control.

What’s included

Package	Description
@microsoft/vally	Core library — discovery, parsing, grading, scoring, reporting
@microsoft/vally-cli	CLI — lint, eval, grade, and more
@microsoft/vally-server	Analytics server — REST API, dashboards, agent integration