Chapter 1 of 6 · Scope and outcomes

Red teaming, prompt injection, and Defender

Runtime assurance 4 hours in a non-production POC

Chapter 1 of 6

Session scope#

What we will do#

Objective. Confirm that a remediated agent version reduces attack success without hiding a regression in any single risk category, and that Defender can route a real alert to the SOC.

Run the same approved attack plan against immutable baseline and remediated versions of the nonproduction policy assistant, then confirm the Defender-to-SOC route. The comparison must show lower overall attack success rate (ASR), no regression in any evaluator, risk category, or attack strategy, and zero prohibited-action success. SOC delivery is reported as a separate result.

Why it matters#

Problem. A better average attack-success rate can hide a worse result in one risk category, and a broken alert route leaves the security team blind right after remediation.

Solution. The per-risk comparison catches a category regression that an average would hide, and the route check confirms Defender can reach the team that must respond.

Boundaries#

Use the authorized nonproduction Foundry project, synthetic inputs, and the read-only get_policy tool; existing tool and backend controls must independently deny prohibited writes, and a model refusal is not the write boundary. Foundry keeps taxonomy and run detail; Defender and the SOC system keep security records; the approved change system keeps the authorization and residual-risk decision.

This session does not authorize production promotion, write-capable testing, or Defender blocking-rule changes. The controlled promotion workflow reads the residual-risk decision the threat-defense control produces before it promotes a version.

Session preparation

Who should join

  • Microsoft Foundry agent and AI safety engineers
  • Cloud security and Microsoft Defender administrators
  • Security operations and AI application owners

What you need

  • A governed nonproduction agent is ready for authorized testing: the platform inventory identifies immutable baseline and remediated versions, the gateway and tool owners confirm the read-only path and blocked prohibited write, and the quality owner retrieves a passing release-gate result. (Sessions 01, 02, 03, 04, 05, 06, 07, 08, and 09.)
  • The approved nonproduction policy assistant has immutable baseline and remediated versions. Its stable endpoint stays on the previously approved version.
  • The get_policy path is read-only. The prohibited write is absent from the MCP allowlist and backend role, or the backend denies it.
  • The project managed identity and red-team operator have Foundry User on the exact Foundry project. That project has an approved judge model and budget.
  • The Session 09 gate remains runnable against the remediated version and returns PASS for its approved record.
  • The security owner has authorized the exact project, agent, versions, attack categories, synthetic-data boundary, run window, stop contact, and same-day region support check.
  • Defender for Cloud AI services protection and the approved Defender-to-SOC route are operating.
  • The SOC owner has accepted an authorized event or route-health result with the source, route, destination, references, observed time, and agent or model context.

Session 10

Red teaming, prompt injection, and Defender slide deck