Chapter 1 of 6
Session scope#
What we will do#
Objective. Confirm that a remediated agent version reduces attack success without hiding a regression in any single risk category, and that Defender can route a real alert to the SOC.
Run the same approved attack plan against immutable baseline and remediated versions of the nonproduction policy assistant, then confirm the Defender-to-SOC route. The comparison must show lower overall attack success rate (ASR), no regression in any evaluator, risk category, or attack strategy, and zero prohibited-action success. SOC delivery is reported as a separate result.
Why it matters#
Problem. A better average attack-success rate can hide a worse result in one risk category, and a broken alert route leaves the security team blind right after remediation.
Solution. The per-risk comparison catches a category regression that an average would hide, and the route check confirms Defender can reach the team that must respond.
Boundaries#
Use the authorized nonproduction Foundry project, synthetic inputs, and the read-only get_policy tool; existing tool and backend controls must independently deny prohibited writes, and a model refusal is not the write boundary. Foundry keeps taxonomy and run detail; Defender and the SOC system keep security records; the approved change system keeps the authorization and residual-risk decision.
This session does not authorize production promotion, write-capable testing, or Defender blocking-rule changes. The controlled promotion workflow reads the residual-risk decision the threat-defense control produces before it promotes a version.
Session preparation
Who should join
- Microsoft Foundry agent and AI safety engineers
- Cloud security and Microsoft Defender administrators
- Security operations and AI application owners
What you need
- A governed nonproduction agent is ready for authorized testing: the platform inventory identifies immutable baseline and remediated versions, the gateway and tool owners confirm the read-only path and blocked prohibited write, and the quality owner retrieves a passing release-gate result. (Sessions 01, 02, 03, 04, 05, 06, 07, 08, and 09.)
- The approved nonproduction policy assistant has immutable baseline and remediated versions. Its stable endpoint stays on the previously approved version.
- The
get_policypath is read-only. The prohibited write is absent from the MCP allowlist and backend role, or the backend denies it. - The project managed identity and red-team operator have Foundry User on the exact Foundry project. That project has an approved judge model and budget.
- The Session 09 gate remains runnable against the remediated version and returns PASS for its approved record.
- The security owner has authorized the exact project, agent, versions, attack categories, synthetic-data boundary, run window, stop contact, and same-day region support check.
- Defender for Cloud AI services protection and the approved Defender-to-SOC route are operating.
- The SOC owner has accepted an authorized event or route-health result with the source, route, destination, references, observed time, and agent or model context.