Chapter 1 of 6
Session scope#
What we will do#
Objective. Give the release owner a pass/block decision that an averaged evaluation score would hide.
Run release-gate.py against two fixed versions of the governed agent, both scored on the same synthetic dataset and recorded Foundry evaluation definition. The approved version must return PASS; an injected tool-process regression must return BLOCK.
Why it matters#
Problem. An averaged evaluation score can pass a release even when one tool path or safety metric has regressed.
Solution. The gate scores final-answer quality, tool process, and safety as separate blocking layers, so a failure in one layer can't hide behind a good average. The controlled promotion workflow runs the same command before promotion.
Boundaries#
The run stays inside the approved nonproduction Foundry project, with the stable endpoint pinned to the approved version throughout. Foundry keeps row-level evaluation detail; the approved release platform keeps payload-free aggregates, gate state, and promotion decisions.
The gate decides release eligibility; it doesn't promote a version, and the MCP tool security control stays the authorization boundary for prohibited writes. The observability and cost controls compare production signals against this baseline and returns a version here after material drift.
Session preparation
Who should join
- Microsoft Foundry agent and evaluation engineers
- AI safety, quality, and application owners
- Release owners responsible for agent promotion
What you need
- The platform inventory names the exact nonproduction Foundry resource, project, policy-assistant agent, approved version, candidate version, and stable-endpoint selector. The platform owner confirms that its stable endpoint selects the approved version. (Sessions 03, 04, 05, and 08.)
- The stable endpoint selects the approved version. The Session 08
get_policypath reads successfully, and the prohibited write is absent or denied. - The evaluation operator and Foundry project managed identity have Foundry User on the exact Foundry project. A network-isolated project also has an approved delegated evaluation subnet.
- The quality owner records a same-day region and evaluator support check in the release policy because no stable support-discovery API is assumed.
- The judge-model deployment, evaluation budget, release owner, quality owner, tool owner, safety owner, cost owner, and exception authority are approved.