Skip to content

Ask Waza

Ask about Waza commands, writing evals, or troubleshooting an error. Paste YAML below for a schema check.

AI answers can be wrong. Do not share secrets or sensitive information. History may be retained.

CLI reference | Privacy & limits

    Check pasted YAML (optional)

    Releases

    Waza v0.39.0: test the workflow, inspect the evidence explains scenario-first evals, preflight, dependency recovery, recorded evidence, independent judging, Dashboard Lab, and controlled comparisons, with screenshots. The article describes the v0.39.0 release scope; published artifact availability is listed separately below.

    Introducing Dashboard Lab: from evaluation scores to evidence takes a closer look at waza serve --lab: finding failures, inspecting task evidence, comparing saved runs, and sharing reports.

    Current published standalone release: v0.39.0

    Section titled “Current published standalone release: v0.39.0”

    Released: October 20, 2026.

    Verified against the standalone release. The installers select published standalone releases, not unreleased main. Run waza update to upgrade an existing standalone installation.

    The following workflows are included in the published standalone v0.39.0. Their opt-in, platform, and evidence boundaries still apply.

    Capability Availability Platform and scope
    new eval --scenario repository/CLI/MCP v0.39.0 Ordinary Linux/macOS/Windows execution; exact eval schema 2.0; CLI needs Git, MCP template is harness-only
    preflight v0.39.0 Agent-free inventory, no runtime/auth/quality certification
    Recorded reliability and serve --lab v0.39.0, Lab opt-in Recorded observations and prototype views, not new live execution authority
    Ordinary run/grade evidence and selected-file regrade v0.39.0, opt-in capture Selected-file private publication on Linux/macOS; unsupported Windows publication is an explicit error; historical/metadata-only readers are separate
    assure authored-case profiles v0.39.0, explicitly selected Bounded native file/output and offline cases; strict preserved-file publication has platform limits; private preserved-native APIs are not default reader authority
    assure calibrate v0.39.0, separate opt-in Reviewed inputs and explicit paid-call acknowledgment required; no default paid calibration
    Controlled strict collection/release policy v0.39.0, explicit policy Offline mock by default; explicit fixed-model, single-turn Copilot SDK native-text profile may incur provider usage; no skills, deny-all client tools (not server isolation), no assurance/runtime-version/currency claim; native durable creation Linux/macOS only
    azd extension commands Separate v0.39.0 artifacts Published separately; registry availability follows the release version-sync PR

    Build main with Go 1.26+, Git LFS, Make, Node.js and npm:

    Terminal window
    git clone https://github.com/microsoft/waza.git
    cd waza
    git lfs install
    git lfs pull
    make build
    ./waza --version

    make build generates dashboard assets before embedding. Bare go install does not fetch LFS artifacts or build the dashboard. See Quick Start for offline validation before credentials.

    Scenario evals now support deterministic finite MCP/CLI fault sequences and report configured dependency modes. These fixtures test agent behavior, not real CLI/server behavior or host isolation. Universal release authority, a full human-labelled corpus, sandbox enforcement and default paid calibration remain out of scope.


    Released: 2026-07-16

    • JSON schema ambiguity detection — extract_json rejects output containing a JSON code block plus an additional JSON document.

    Released: 2026-07-16

    • JSON schema output extraction — json_schema supports extract_json: true to validate a single JSON document wrapped in prose or a Markdown JSON code block while preserving strict validation by default.
    • APM-installed skill detection — Skill discovery recognizes skills installed by Azure API Management (#402).
    • Dependencies and tooling — Updated the Copilot SDK, OpenTelemetry, azd, documentation tooling, dashboard dependencies, and GitHub Actions.

    Released: 2026-06-30

    • Focused eval suggestions — waza suggest can target specific coverage gaps with --count, --focus, --dry-run, --apply, and --force (#357, #380).
    • Per-turn checkpoint graders — Task YAML can assert intermediate multi-turn state with checkpoints[] (#358, #386).
    • Rubric preset library — Prompt graders can reuse built-in rubrics for groundedness, helpfulness, instruction following, refusal correctness, and tool-use appropriateness (#360, #381).
    • Spec verification — waza spec verify checks that eval tasks cover the executable contract in SKILL.md (#361, #385).
    • OpenTelemetry export — waza run --otel-* flags export eval, task, turn, model-call, and tool-call spans (#362, #383).
    • MCP server mocks — eval.yaml can define mcp_mocks: for hermetic Copilot SDK evals (#363, #387).
    • Regression gates — waza gate compares current results with a baseline and exits 0/1/2/3 for pass, regression, golden failure, or config error (#364, #384).
    • Adversarial harness — waza adversarial runs prompt-injection and scope-bypass packs, optionally configured by adversarial: in eval YAML (#365, #392).
    • Tool metrics and argument matchers — Results include normalized tool_events[]; graders can use structured expect_tools[].args and tool_calls.expect[].args matchers (#366, #388).
    • Snapshot and replay — waza run --snapshot captures task snapshots and waza replay checks them offline or bisects divergent runs (#367, #391).
    • Schema semver policy — schemaVersion uses MAJOR.MINOR compatibility rules, documented in the schema changelog (#368, #382).
    • Dashboard SSE resume — /api/v1/runs/{runId}/events and legacy /api/events support Last-Event-ID and lastEventId resume (#178, #397).
    • Internal refactor — Phase 1 internal cleanup landed without user-facing CLI or schema changes (#10).

    Released: 2026-06-06

    • Copilot SDK v1.0.0 upgrade — Upgraded to Copilot SDK v1.0.0 and the dashboard now surfaces premium-request credits.
    • Model-aware dashboard pricing — Dashboard cost calculation applies per-model pricing for more accurate run costs.
    • Git worktree resources in task inputs — Tasks can reference git worktree resources as inputs.

    Pre-built binaries for all major platforms:

    Platform Architecture Download
    macOS Apple Silicon (ARM64) waza-darwin-arm64
    macOS Intel (AMD64) waza-darwin-amd64
    Linux x64 (AMD64) waza-linux-amd64
    Linux ARM64 waza-linux-arm64
    Windows x64 (AMD64) waza-windows-amd64.exe
    Windows ARM64 waza-windows-arm64.exe

    Terminal window
    curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash

    On Windows, use this command only from Git Bash, MSYS2, or Cygwin. If PowerShell pipes to WSL’s bash.exe, the script installs the Linux binary inside WSL.

    Verify the installation:

    Terminal window
    waza --version

    Install waza as an Azure Developer CLI extension.

    The published extension release is 0.39.0, separate from standalone v0.39.0. Registry availability follows the release version-sync PR; publishing artifacts alone does not update the registry on main. Check the installed version after adding the source:

    Terminal window
    azd ext source add -n waza -t url -l https://raw.githubusercontent.com/microsoft/waza/main/registry.json
    azd ext install microsoft.azd.waza
    azd waza --version
    azd waza --help

    Use commands supported by that installed artifact. Check per-command help and the operation’s platform limits rather than merely substituting azd waza for waza. Some historical registry entries are owned by a different release repository; their presence is not a live artifact-availability verification. For supported existing evals:

    Terminal window
    azd waza run eval.yaml

    For older versions, changelogs, and pre-release builds, see the full GitHub Releases page.