Chapter 6 of 6 · Validation and operations

Continuous evaluation and Foundry platform alerts

Optional module · Draft 3.5 hours

Chapter 6 of 6

Confirm the result#

The module is complete when:

  • the diagnostic setting exists on the Foundry account and points to the approved Log Analytics workspace;
  • the action group is enabled and owned by the operating team;
  • the throttling, latency, and token-volume metric alert rules are enabled, tagged with implementationSession=optional-module-continuous-evaluation-platform-alerts, and scoped to the approved Foundry account;
  • the continuous evaluation rule is enabled for the governed agent; and
  • recent evaluation runs show status after sampled traffic arrives.

Stop if diagnostic rows never arrive, any alert is disabled, the action group has the wrong owner, or evaluation runs remain absent after traffic and the sampling cap should allow them.

After implementation#

What remainsOwner
Diagnostic setting, alert rules, action group, and thresholdsObservability owner
Continuous evaluation rule, evaluator set, thresholds, and sampling capAI quality owner
Foundry account, project, managed identity, and role assignmentPlatform owner
Incident route and restore decisionService owner
KQL checks and alert tuning notesAzure Monitor operations owner

Restore through the owning change paths:

  1. Disable only the noisy module-owned alert rule while the owner corrects the threshold.
  2. Disable the continuous evaluation rule before deleting it if evaluator cost or preview behavior becomes a problem.
  3. Remove the diagnostic setting only after the observability owner confirms another route keeps the required platform logs.
  4. Delete the action group only when no alert rule still uses it.
  5. Keep Application Insights, Log Analytics, incident records, and release evaluation gates in place.

Optional module

Continuous evaluation and Foundry platform alerts slide deck