Autonomous SRE with Azure SRE Agent
Ten hands-on challenges: configure SRE Agent building blocks (skills, subagents, connectors, scheduled tasks), then trigger and observe 6 autonomous incident scenarios. Finish with a capstone where you build your own production-ready SRE agent.
Challenges
Challenges 00–06 cover SRE Agent configuration primitives. Challenges 07–09 establish observability baselines. Challenges 10–15 are live incident scenarios spanning PaaS, IaaS, and network layers. Challenges 16–18 are advanced operational skills and automation tasks. Challenge 19 is the capstone: bring your own use case and build a production-ready agent. Coaches: press Shift+C to unlock solution guides.
Prerequisites &
Lab Setup
Deploy the lab workload infrastructure with Terraform, then create your own Azure SRE Agent resource via the portal or CLI — an empty agent with zero configuration. Generate baseline traffic to seed the observability pipeline before Challenges 01–06.
Connect Your
Codebase
Your agent is running but isolated from source code. Connect a GitHub OAuth Connector and link your repository so the agent can read code and open pull requests.
Explore the
Knowledge Base
Your agent answers from training data alone — it knows nothing about your lab. Attach knowledge documents so it gives grounded, accurate answers about your environment.
Discover Operational
Skills
Your agent can answer questions but can't act. Author a skill YAML so it can execute KQL queries, inspect Container App health, and run operational runbooks.
Discover Connected
Systems
Your agent relies on stale training data for best practices. Connect MCP servers to give it access to live, curated knowledge and tools beyond its training cutoff.
Discover Specialist
Agents
A generalist handles everything from one system prompt. Define specialist subagents — each with its own identity, tools, and runbooks — and route incidents to the right expert.
Understand
Response Plans
Everything so far is chat-driven — you ask, it answers. Configure incident filters and response plans so the agent acts autonomously when alerts fire at 3 AM.
Hybrid Ecosystem
Telemetry
The Parking Manager spans Container Apps, VMs, and a third-party OpenTelemetry API. Use the agent to query all three platforms and produce a unified cross-system view.
Application Dependency
Mapping
When an app fails, which dependency is the culprit? Drive the agent to map the full Parking Manager dependency graph using App Insights — the foundation for every root-cause investigation.
Daily Application
Health Report
Ask the agent to generate a structured health report for all four Parking Manager APIs — CPU, memory, error rate, response time — with a health score and recommended remediation for any service falling below threshold.
Incident to
GitHub Issue
Trigger a Parking Manager incident and let the agent investigate, then close the loop by automatically opening a GitHub issue with the root cause, timeline, and remediation steps — no copy-paste required.
Guest OS Failure
Investigation
Stop NGINX inside the web-tier VMs and watch the agent discover the failure from Syslog telemetry, assess the full blast radius across both VMs, and restore the service autonomously on every affected instance.
Network Security
Investigation
Inject an NSG misconfiguration that silently drops database traffic. The agent finds the blocking rule from flow log evidence and remediates it autonomously — no application-level error message to guide it.
Routing Failure
Investigation
Introduce an asymmetric routing black hole where return packets never arrive.
Drive a diagnostic conversation with the network-traffic-analyst
over effective routes and next-hop analysis to find and fix the return path.
Application Root
Cause Analysis
From a 5xx spike the agent traces through container logs, Application Insights telemetry, and source code to the exact line causing an OOM crash — then opens a GitHub issue linking evidence to code. No human switches tools.
Autonomous
Remediation
Trigger a Parking Manager backend failure, then watch the agent detect it, apply the automated fix, and validate recovery — closing the full remediation loop without human intervention.
Daily Network
Health Report
Create a scheduled task that reviews 24 hours of network telemetry every morning and produces a decision-ready summary: denied flows, top talkers, missing VNet coverage, and unusual ports.
Observability
Freshness Verification
Build a scheduled task that proves the monitoring pipeline itself is alive — checking VNet Flow Log ingestion end-to-end so silent telemetry loss is caught before it masks real failures.
Subscription Cost
Optimization Review
Run a subscription-wide cost review: resource inventory, actual spend, utilization, and Azure Advisor recommendations — grounded in business context so the savings table is actionable, not just technically correct.
Build Your Own
Production-Ready SRE Agent
Bring a real reliability problem from your own environment. Design and implement a custom subagent — skills, knowledge docs, incident filter or scheduled trigger — then demo it end-to-end to the group and defend its production readiness.
Subagents
Each subagent is a specialist with its own system prompt, tool grants, and incident routing.
The network-traffic-analyst supports both interactive and autonomous modes.
Solution Guides
Mini-lectures, expected agent behavior, common failure modes, and debrief discussion guides. See the Coach README for the full index.