RVAP logo

AI Governance Co-implementation · Session 06

Azure API Management AI gateway design and implementation

270 minutes · Record the design, deploy one controlled route, and check its ingress boundary

Why it matters

Problem. An unrecorded gateway change can drift from what was approved, and forwarding the
caller's own credential to Foundry would let a compromised client reach the agent directly.

Solution.

  • The design record locks scope, backend, identity, network, safety, and restore decisions before
    deployment.
  • APIM calls Foundry with its own managed identity, never the caller's credential.
  • Deployment preflight and ARM what-if confirm the preview touches only the marked scope.
  • APIM returns 401 Unauthorized for an invalid bearer token before Content Safety or Foundry ever
    sees it.

Architecture and control boundary

The caller sends an Entra application token and APIM subscription key. APIM validates both, applies
limits and Content Safety, then uses its managed identity for the pinned Foundry agent.

Application Insights receives correlation and token metrics. No request or response body is logged.

APIM validates the caller and applies limits and safety checks before its managed identity reaches the Foundry agent.

Implementation tradeoffs

Decision Required answer
Scope Approved nonproduction APIM resource group and change record
Backend Pinned Foundry agent, endpoint reference, and managed-identity access
Network APIM, Foundry, and Content Safety paths agree with the design record
Safety Approved Content Safety backend and threshold
Restore Named route disable or rollback path

Keep the current APIM route when a decision or readiness gap is open.

Working path

Total session: 270 minutes. Guided work: about 210 minutes.

  1. Complete gateway-design-record.json and the deployment inputs.
  2. Run combined preflight. It checks the record before Azure and ARM what-if.
  3. Review the preview and stop if it reaches APIM itself or an unrelated resource.
  4. Deploy the marked API, product, backend pool, policy, and diagnostics.
  5. Send the synthetic invalid-token request once.

Reserve time for decisions, preview review, and the operating handoff.

Required controls and access

Control Required value
Operator Time-bound Contributor on the exact APIM resource group
APIM identity Foundry Agent Consumer on the individual agent
Content Safety identity Cognitive Services User on the exact resource
Client Approved Entra application identity and workload subscription
Telemetry Correlation and token metrics; zero body logging
Retry One read-safe retry; secondary backend disabled

Confirm, operate, and hand off

Confirm once

Use a valid workload subscription key with an invalid bearer token. APIM must return 401 before
Content Safety or Foundry receives the request.

Keep in operation

The delivery owner maintains the design record. The platform owner maintains the deployment
configuration. The product, identity, safety, and operations owners maintain their controls.

API Center and MCP inventory guide records the route in API Center. MCP tool security guide adds the MCP tool boundary.

Thank you!

Notes: Work on one nonproduction Foundry Agent Service policy-assistant route.

Notes: The invalid-token check is the one standard-mode result.

Notes: APIM owns the live route. Foundry owns the agent.

Notes: Design preflight runs before Azure preflight.

Notes: Do not enter endpoint URLs, keys, tokens, prompts, responses, or customer data in source control.

Notes: Stop if the retry can repeat a consequential action.

Notes: Restore through the approved APIM path. Remove only marked APIM gateway child resources.