Chapter 2 of 6
Architecture at a glance#
The flow has three parts.
First, Azure Policy stages the approved cost-tag requirement on resource groups across the approved subscription and uses a managed identity to inherit those values to child resources. Second, Cost Management budget notifications watch the approved AI resource groups and meter categories. Third, API Management emits token metrics with cost-center and consumer dimensions while a per-product quota limits token use.
The repository owns the Bicep, APIM policy fragment, rate table, and KQL query. Azure holds the live policy assignments, budget, APIM policy, token metrics, and billed cost. Finance owns the rate table and reconciliation cadence.
Design choices and tradeoffs#
| Decision | Chosen approach | Benefits | Costs and limitations |
|---|---|---|---|
| Allocation anchor | Stage required tags on resource groups and inherit them to resources | Billed cost can be grouped by the same finance tags across services | The tag requirement is subscription-wide and stays in DoNotEnforce until approved |
| Budget signal | Filter by resource group and meter category, then notify email and an action group | Cost owners see actual and forecasted spend changes without broad subscription noise | Budgets notify only; they do not stop resources |
| Token signal | Emit APIM token metrics with five low-cardinality dimensions from trusted gateway context | FinOps can attribute token use before billed cost lands | High-cardinality values are dropped by metric limits |
| Quota | Per-product monthly token quota and tokens-per-minute rate | Product owners get a hard gateway boundary for shared routes | Quotas apply at the gateway counter key, not to direct model endpoint calls |
| Rate table | Finance-owned JSON table, used by KQL and reporting | Rates are reviewable and versioned | Estimated cost must be reconciled with Cost Management |
Architecture guidance#
Use Azure Policy assignment structure for assignment identity, metadata, parameters, and enforcement mode.
Use Create a budget with Bicep for budget notifications, filters, forecasted thresholds, and the budget boundary.
Use llm-emit-token-metric for token metrics, custom-dimension limits, Application Insights prerequisites, and streaming token-count caveats.