Chapter 4 of 6 · Decisions and boundaries

Azure API Management AI gateway design and implementation

Runtime assurance 4.5 hours in a non-production POC

Chapter 4 of 6

Decisions and stop conditions#

Resolve every __REQUIRED_*__ value in the design record, gateway-control.json, and sandbox.json. Use an endpoint reference in the design record, never the live endpoint.

Set recordStatus to ready-for-implementation when no readiness gap is open. Each gap needs an owner and a resolution action. The record must name foundry-agent-service and the policy-assistant-responses variant. Its APIM instance must match sandbox.json.

ControlApproved value
Tokens per minute per APIM subscription20,000
Daily token quota per APIM subscription500,000
Request body limit65,536 bytes
Backend response-header timeout120 seconds
Retry for 429/5xx1
Circuit breaker5 errors in 1 minute; open for 1 minute

APIM enables Prompt Shields and checks Hate, SelfHarm, Sexual, and Violence at threshold 4. It logs zero request and response body bytes and no client IP. Keep the secondary backend disabled.

Stop before deployment when the design record is incomplete, a resource differs from approved inputs, a required role or workload subscription is absent, Content Safety uses a key, or ARM what-if changes APIM itself or an unrelated resource. Stop if a retry could repeat a consequential action. Do not place live endpoint or customer data in source control.

Session 06

Azure API Management AI gateway design and implementation slide deck