Skip to content

Ask Waza

Ask about Waza commands, writing evals, or troubleshooting an error. Paste YAML below for a schema check.

AI answers can be wrong. Do not share secrets or sensitive information. History may be retained.

CLI reference | Privacy & limits

    Check pasted YAML (optional)

    Releases

    Waza v0.41.0: recover durable work, keep the evidence explains execution attempts and lineage, checkpoints, lifecycle semantics, evidence states, standalone aggregation, the optimizer adapter contract and native question callbacks, with the release’s publication record and limits.

    Waza v0.40.1: make release workflows match their contracts explains the selected-task coverage, strict token-budget exit, grouping, hook, saved-run, artifact-path, execution-startup, and JavaScript-grader hotfixes.

    Waza v0.40.0: run exactly the work you declared explains exact work units, input-derived identities, terminal outcomes, the container reference, and practical evidence improvements, including the published release’s known limits.

    Waza v0.39.0: test the workflow, inspect the evidence explains scenario-first evals, preflight, dependency recovery, recorded evidence, independent judging, Dashboard Lab, and controlled comparisons, with screenshots. The article describes the v0.39.0 release scope; published artifact availability is listed separately below.

    Introducing Dashboard Lab: from evaluation scores to evidence takes a closer look at waza serve --lab: finding failures, inspecting task evidence, comparing saved runs, and sharing reports.

    Current published standalone release: v0.41.0

    Section titled “Current published standalone release: v0.41.0”

    Released: November 13, 2026.

    The standalone release is published, with six platform binaries and checksums.txt. The installers select published standalone releases, not unreleased main. Run waza update to upgrade an existing standalone installation. See the full comparison and the v0.41.0 changelog entry. The release assets expose GitHub-computed SHA-256 digests in checksums.txt; publication does not by itself establish execution on every platform. Unlike the historical v0.40.1 tags, the v0.41.0 tag carries 0.41.0 in version.txt and extension.yaml.

    Released: November 6, 2026.

    The v0.40.1 standalone release was published with six platform binaries and checksums.txt. See the full comparison and the v0.40.1 release article. The successful release workflow built all six standalone platforms and published checksums.txt. The v0.40.1 tag resolves to e0c251df896bee4e0f9b38a693a6105bf7eadd19.

    The following guides describe standalone v0.40.0. The azd extension artifact and registry sync are independent gates; a standalone release does not establish extension availability. The v0.40.0 tag resolves to 93e80d7fdcbf9206215366d6ecca7ed08d26b2b3; a tag alone is not asset publication. See the runner contract and exact work-unit guide, logical identities, terminal outcomes, local compatibility and migration, and the local container foundation. Runner execution supports the documented mock/text and explicitly opted-in Copilot/text profiles, with tools and skills disabled for Copilot execution. Admitted eval metrics are identity-bound, uncomputed metadata, not aggregate values or threshold decisions. Paid native HTTP execution is local-only; see the peer, Host and origin policy. General native file/tool/skill workflows and full external task/suite interoperability remain incomplete. Exact-unit deadlines and cancellation do not fix the separate legacy multi-trial timeout issue #865. The later timeout/persistence fix #870 is post-v0.40.0 source behavior, not included in the published v0.40.0 artifacts. The container guide distinguishes the default mock-only scratch target from the native-capable exact-unit Debian/glibc target, which is tested with the actual bundled CLI and a local synthetic provider without external runtime networking. Local image IDs are not registry pins; the published native reference passed digest-pinned acceptance on Linux AMD64. The manual-only GHCR publisher and separate verifier have independent gates. The published image record identifies its immutable digest, source and declared Linux AMD64 platform. Publisher attempt 1 and the verifier succeeded: anonymous pull, source/platform, signed provenance/finalization and mock/native synthetic-provider production controls passed for that immutable digest (1/1 test, 0 skipped). Independent infrastructure-owner review was waived for this one publication, not performed. A publisher run alone does not establish public-image acceptance. See the public-digest verification procedure. See roadmap #754 for the remaining release requirements.

    The selected AgentV assertion adapter adds offline, version-pinned waza interop import/export for one case-sensitive contains assertion and its single-check native grader mapping. Whole-task/suite imports, AgentEvals-spec/Harbor/ASSERT adapters and full evidence/provenance round trips remain unsupported; #789 remains open. This is assertion conformance, not whole-suite or agent-quality evidence.

    The release also includes task-level MCP mocks, program-grader context, host lifecycle evidence, and Lab conversations. v0.40.0 result writers emit 1.8; v0.39.1 writes 1.7. Missing versions in legacy eval/result inputs still default to 1.4. Runner request and terminal schemas remain independently versioned 1.0; task YAML has no independent version field. See schema versions.

    waza optimize adds the versioned 1.0 optimizer/evaluator contract. It includes capability negotiation, content-addressed candidates, suite pinning, held-out qualification, bounded budgets, measured or unavailable spend, and explicit failure categories. It ships offline fake, external command and paid copilot-reflect adapters. Existing runs and result schemas are unchanged, and Lab does not display optimizer reports. See #790.

    The following workflows were introduced in v0.39.0 and remain included in v0.41.0. Their opt-in, platform, and evidence boundaries still apply.

    Capability Availability Platform and scope
    new eval --scenario repository/CLI/MCP v0.39.0 Ordinary Linux/macOS/Windows execution; exact eval schema 2.0; CLI needs Git, MCP template is harness-only
    preflight v0.39.0 Agent-free inventory, no runtime/auth/quality certification
    Recorded reliability and serve --lab v0.39.0, Lab opt-in Recorded observations and prototype views, not new live execution authority
    Ordinary run/grade evidence and selected-file regrade v0.39.0, opt-in capture Selected-file private publication on Linux/macOS; unsupported Windows publication is an explicit error; historical/metadata-only readers are separate
    assure authored-case profiles v0.39.0, explicitly selected Bounded native file/output and offline cases; strict preserved-file publication has platform limits; private preserved-native APIs are not default reader authority
    assure calibrate v0.39.0, separate opt-in Reviewed inputs and explicit paid-call acknowledgment required; no default paid calibration
    Controlled strict collection/release policy v0.39.0, explicit policy Offline mock by default; explicit fixed-model, single-turn Copilot SDK native-text profile may incur provider usage; no skills, deny-all client tools (not server isolation), no assurance/runtime-version/currency claim; native durable creation Linux/macOS only
    azd extension commands Registry through v0.41.0; v0.41.0 artifact published separately Registry/version metadata synchronized; check the installed extension version

    Build the tagged release source with Go 1.26+, Git LFS, Make, Node.js 22.23.4+ (22.x) or 24+ (recommended), and npm 9.6.5+:

    Terminal window
    git clone --branch v0.41.0 https://github.com/microsoft/waza.git
    cd waza
    git lfs install
    git lfs pull
    make build VERSION=0.41.0
    ./waza --version

    make build generates dashboard assets before embedding. Bare go install does not fetch LFS artifacts or build the dashboard. See Quick Start for offline validation before credentials. The v0.41.0 tag resolves to d537bf00d4d442a1935ceba2fbc664b9ccb497b2 and already carries 0.41.0 in version.txt and extension.yaml, unlike the historical v0.40.1 tags. The Makefile defaults to VERSION=0.1.0; pass VERSION=0.41.0 for this source build. Official release binaries use the tag version injected with linker flags. This local version label does not verify asset publication or extension/registry availability.

    Scenario evals now support deterministic finite MCP/CLI fault sequences and report configured dependency modes. These fixtures test agent behavior, not real CLI/server behavior or host isolation. Universal release authority, a full human-labelled corpus, sandbox enforcement and default paid calibration remain out of scope.


    Released: October 26, 2026.

    • waza dev accepts a workspace skill name, directory, or SKILL.md file in iterative, Copilot-report, and trigger-scaffolding modes, including Windows paths. Bare names keep workspace precedence; batch names and --all are unchanged (#747).
    • Documentation examples, Ask Waza configuration, and dashboard build dependencies are included in the full comparison.

    Released: 2026-07-16

    • JSON schema ambiguity detection — extract_json rejects output containing a JSON code block plus an additional JSON document.

    Released: 2026-07-16

    • JSON schema output extraction — json_schema supports extract_json: true to validate a single JSON document wrapped in prose or a Markdown JSON code block while preserving strict validation by default.
    • APM-installed skill detection — Skill discovery recognizes skills installed by Azure API Management (#402).
    • Dependencies and tooling — Updated the Copilot SDK, OpenTelemetry, azd, documentation tooling, dashboard dependencies, and GitHub Actions.

    Released: 2026-06-30

    • Focused eval suggestions — waza suggest can target specific coverage gaps with --count, --focus, --dry-run, --apply, and --force (#357, #380).
    • Per-turn checkpoint graders — Task YAML can assert intermediate multi-turn state with checkpoints[] (#358, #386).
    • Rubric preset library — Prompt graders can reuse built-in rubrics for groundedness, helpfulness, instruction following, refusal correctness, and tool-use appropriateness (#360, #381).
    • Spec verification — waza spec verify checks that eval tasks cover the executable contract in SKILL.md (#361, #385).
    • OpenTelemetry export — waza run --otel-* flags export eval, task, turn, model-call, and tool-call spans (#362, #383).
    • MCP server mocks — eval.yaml can define mcp_mocks: for hermetic Copilot SDK evals (#363, #387).
    • Regression gates — waza gate compares current results with a baseline and exits 0/1/2/3 for pass, regression, golden failure, or config error (#364, #384).
    • Adversarial harness — waza adversarial runs prompt-injection and scope-bypass packs, optionally configured by adversarial: in eval YAML (#365, #392).
    • Tool metrics and argument matchers — Results include normalized tool_events[]; graders can use structured expect_tools[].args and tool_calls.expect[].args matchers (#366, #388).
    • Snapshot and replay — waza run --snapshot captures task snapshots and waza replay checks them offline or bisects divergent runs (#367, #391).
    • Schema semver policy — schemaVersion uses MAJOR.MINOR compatibility rules, documented in the schema changelog (#368, #382).
    • Dashboard SSE resume — /api/v1/runs/{runId}/events and legacy /api/events support Last-Event-ID and lastEventId resume (#178, #397).
    • Internal refactor — Phase 1 internal cleanup landed without user-facing CLI or schema changes (#10).

    Released: 2026-06-06

    • Copilot SDK v1.0.0 upgrade — Upgraded to Copilot SDK v1.0.0 and the dashboard now surfaces premium-request credits.
    • Model-aware dashboard pricing — Dashboard cost calculation applies per-model pricing for more accurate run costs.
    • Git worktree resources in task inputs — Tasks can reference git worktree resources as inputs.

    Pre-built binaries for all major platforms:

    Platform Architecture Download
    macOS Apple Silicon (ARM64) waza-darwin-arm64
    macOS Intel (AMD64) waza-darwin-amd64
    Linux x64 (AMD64) waza-linux-amd64
    Linux ARM64 waza-linux-arm64
    Windows x64 (AMD64) waza-windows-amd64.exe
    Windows ARM64 waza-windows-arm64.exe

    Terminal window
    curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash

    On Windows, use this command only from Git Bash, MSYS2, or Cygwin. If PowerShell pipes to WSL’s bash.exe, the script installs the Linux binary inside WSL.

    Verify the installation:

    Terminal window
    waza --version

    Install waza as an Azure Developer CLI extension.

    The published extension release is 0.41.0, published November 13, 2026 with six platform archives, separate from the standalone release. The repository registry lists through 0.41.0. The release workflow published both releases and generated registry/version updates on release/v0.41.0; only its final automated PR-creation step failed, so the PR was opened manually. Artifact publication and registry synchronization remain separate gates; publishing archives alone does not update the registry. The installation source below follows main. The v0.41.0 registry update merged in release-sync PR #987. The generated entry at commit c385f6d12d393e9c23c54b2cc19478936604cd86 has exactly six platform URLs, entrypoints and SHA-256 values matching the published archives, with prior registry data unchanged. That registry comparison does not establish execution on all six platforms or prove that an existing installation has updated to v0.41.0. Check the installed version after adding the source:

    Terminal window
    azd ext source add -n waza -t url -l https://raw.githubusercontent.com/microsoft/waza/main/registry.json
    azd ext install microsoft.azd.waza
    azd waza --version
    azd waza --help

    Use commands supported by that installed artifact. Check per-command help and the operation’s platform limits rather than merely substituting azd waza for waza. Some historical registry entries are owned by a different release repository; their presence is not a live artifact-availability verification. For supported existing evals:

    Terminal window
    azd waza run eval.yaml

    For older versions, changelogs, and pre-release builds, see the full GitHub Releases page.