Chapter 2 of 6
Architecture at a glance#
The Foundry account emits platform logs and metrics. The diagnostic setting sends them to the approved Log Analytics workspace. Azure Monitor metric alerts watch three operator-owned signals: 429 throttling, time to last byte, and token volume. The action group routes those alerts.
The Foundry project stores monitoring and evaluation data in Application Insights. The continuous evaluation script creates an evaluation from the recorded evaluator list, then creates or updates a rule for RESPONSE_COMPLETED events from the governed agent. The rule uses maxHourlyRuns as the sampling cap.
Application Insights, Log Analytics, and Foundry remain authoritative for live telemetry and evaluation runs. The repository records the intended state and the owner decisions that live services do not reconstruct later.
Design choices and tradeoffs#
| Decision | Chosen approach | Benefits | Costs and limitations |
|---|---|---|---|
| Platform diagnostics | allLogs and AllMetrics to the approved Log Analytics workspace | Gives operators one query surface for platform logs and exported metrics | Workspace access and retention must match the content risk |
| Alert set | 429 throttling, time to last byte, and token volume | Covers the signals the service owner can tune or escalate | Thresholds need baseline traffic and may change after the first operating week |
| Latency signal | AzureOpenAITTLTInMS, paired with token volume | Avoids the legacy Latency metric that Microsoft warns against for Azure OpenAI | Token-heavy requests can still look slow without being unhealthy |
| Evaluation sampling | Continuous evaluation rule with maxHourlyRuns | Bounds evaluation cost and lets the AI quality owner tune traffic sampling | Recurring evaluation surfaces are preview and need an approved preview-use decision |
| Restore | Disable module-owned alerts or the evaluation rule before deleting resources | Keeps telemetry available while owners repair thresholds or evaluators | Removal still needs the approved Azure and Foundry change paths |
Architecture guidance#
Use Monitor agents with the Agent Monitoring Dashboard for the continuous evaluation rule, Application Insights dependency, preview status, and Foundry User role requirement.
Use Monitoring data reference for Azure OpenAI for the metric names, dimensions, token metrics, throttling signal, and latency guidance.
Use Diagnostic Settings in Azure Monitor for diagnostic setting behavior and Log Analytics as a destination for resource logs and platform metrics.