Agent Workload — Reference Component

Part of the Frontier Fabric AgentOps RVAS. This is the telemetry source you deploy in Challenge 1 — a full-stack Azure AI Foundry agent that emits the traces, custom token/cost metrics, and conversation data the Control Tower later correlates.

Full-stack AI agent runtime with end-to-end observability — from user click to model inference.

Architecture Overview#

The Agents Runtime is the core execution block of the observability platform. It orchestrates user interactions through a gateway, processes requests across three microservices, persists data, and emits telemetry at every layer.

Architecture Diagram#

Copy┌──────────┐
│   User   │
└────┬─────┘
     │ HTTPS
     ▼
┌──────────────────────────────────────────────────────────────┐
│                    API Management Gateway                    │
│            (PTU load balancing · rate limiting)               │
└────┬─────────────────┬──────────────────┬────────────────────┘
     │                 │                  │
     ▼                 ▼                  ▼
┌─────────────────────────────────────────────────────────────┐
│              Container Apps Environment                      │
│                                                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────────────┐ │
│  │  Frontend   │  │   Backend    │  │       Agent          │ │
│  │  (Next.js)  │  │  (FastAPI)   │  │     (FastAPI)        │ │
│  │  Port 3000  │  │  Port 8000   │  │     Port 8001        │ │
│  └─────────────┘  └──────┬───────┘  └──────────┬──────────┘ │
│                          │                      │            │
│                          ▼                      ▼            │
│                   ┌────────────┐       ┌─────────────────┐   │
│                   │ Cosmos DB  │       │ Azure AI Foundry │   │
│                   │ (agentsdb) │       │  (GPT-4o model)  │   │
│                   └────────────┘       └─────────────────┘   │
│                                                              │
│         ┌──────────────────────────────────┐                 │
│         │      Application Insights        │                 │
│         │  (traces · metrics · logs · map) │                 │
│         └──────────────────────────────────┘                 │
└──────────────────────────────────────────────────────────────┘

Prerequisites#

Quick Start#

Copy# Authenticate with Azure
azd auth login

# Initialize the environment (first time only)
azd init

# Provision infrastructure and deploy all services
azd up

After deployment completes, azd prints the service URLs:

Copy  frontend  https://frontend.<env>.<region>.azurecontainerapps.io
  backend   https://backend.<env>.<region>.azurecontainerapps.io
  agent     https://agent.<env>.<region>.azurecontainerapps.io

Services#

Frontend — src/frontend#

Backend — src/backend#

Agent — src/agent#

API Reference#

Backend Service (/api)#

Create a Conversation

CopyPOST /api/conversations
Content-Type: application/json

{
  "title": "New Conversation"
}

Response 201 Created

Copy{
  "id": "conv-abc123",
  "title": "New Conversation",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Get a Conversation

CopyGET /api/conversations/{id}

Response 200 OK

Copy{
  "id": "conv-abc123",
  "title": "New Conversation",
  "messages": [
    {
      "role": "user",
      "content": "Hello!",
      "timestamp": "2024-01-15T10:31:00Z"
    },
    {
      "role": "assistant",
      "content": "Hi there! How can I help you today?",
      "timestamp": "2024-01-15T10:31:02Z"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:31:02Z"
}

Send a Message

CopyPOST /api/conversations/{id}/messages
Content-Type: application/json

{
  "role": "user",
  "content": "Explain observability in distributed systems."
}

Response 200 OK

Copy{
  "role": "assistant",
  "content": "Observability in distributed systems refers to...",
  "timestamp": "2024-01-15T10:32:05Z",
  "usage": {
    "prompt_tokens": 128,
    "completion_tokens": 256,
    "total_tokens": 384
  }
}

Health Check (Backend)

CopyGET /api/health

Response 200 OK

Copy{ "status": "healthy", "service": "backend", "version": "1.0.0" }

Agent Service (/api/agent)#

Invoke Agent (Synchronous)

CopyPOST /api/agent/invoke
Content-Type: application/json

{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is Azure Container Apps?" }
  ],
  "temperature": 0.7,
  "max_tokens": 1024
}

Response 200 OK

Copy{
  "message": {
    "role": "assistant",
    "content": "Azure Container Apps is a serverless container platform..."
  },
  "usage": {
    "prompt_tokens": 64,
    "completion_tokens": 200,
    "total_tokens": 264
  },
  "model": "gpt-4o",
  "duration_ms": 1523
}

Invoke Agent (Streaming)

CopyPOST /api/agent/stream
Content-Type: application/json

{
  "messages": [
    { "role": "user", "content": "Write a haiku about cloud computing." }
  ]
}

Response 200 OK (Server-Sent Events)

Copydata: {"delta": "Servers ", "type": "content"}
data: {"delta": "in the sky,", "type": "content"}
data: {"delta": "\n", "type": "content"}
data: {"delta": "Scaling ", "type": "content"}
data: {"delta": "without limits—", "type": "content"}
data: {"delta": "\n", "type": "content"}
data: {"delta": "Code ", "type": "content"}
data: {"delta": "runs everywhere.", "type": "content"}
data: {"type": "done", "usage": {"prompt_tokens": 32, "completion_tokens": 18, "total_tokens": 50}}

Health Check (Agent)

CopyGET /api/health

Response 200 OK

Copy{ "status": "healthy", "service": "agent", "version": "1.0.0" }

Environment Variables#

Backend Service#

Agent Service#

Frontend Service#

Local Development#

1. Backend#

Copycd src/backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Set required environment variables
export COSMOS_ENDPOINT="https://<your-account>.documents.azure.com:443/"
export COSMOS_DATABASE="agentsdb"
export AGENT_SERVICE_URL="http://localhost:8001"
export APPLICATIONINSIGHTS_CONNECTION_STRING="<your-connection-string>"

uvicorn app:app --reload --port 8000

2. Agent#

Copycd src/agent
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Set required environment variables
export AZURE_OPENAI_ENDPOINT="https://<your-resource>.openai.azure.com/"
export AZURE_OPENAI_DEPLOYMENT="gpt-4o"
export APPLICATIONINSIGHTS_CONNECTION_STRING="<your-connection-string>"

uvicorn app:app --reload --port 8001

3. Frontend#

Copycd src/frontend
npm install

# Create .env.local
echo 'NEXT_PUBLIC_API_BASE_URL=http://localhost:8000' > .env.local

npm run dev

The frontend is available at http://localhost:3000.

Monitoring & Observability#

Application Insights Integration#

All three services emit telemetry through the Azure Monitor OpenTelemetry SDK. Traces, metrics, and logs are collected automatically and correlated across service boundaries.

Distributed Tracing

Every inbound HTTP request generates a trace that flows through the entire call chain:

CopyFrontend → APIM → Backend → Agent → Azure OpenAI

Each span in the trace includes:

• HTTP method, URL, and status code
• Duration and dependency details
• Custom attributes (conversation ID, model name, token counts)

View traces: Azure Portal → Application Insights → Transaction search → filter by operation name.

Application Map

The Application Map automatically discovers service dependencies and displays:

• Request rates and failure rates between services
• Average response times per dependency
• Cosmos DB and Azure OpenAI as external dependencies

View the map: Azure Portal → Application Insights → Application map.

Custom Metrics

The agent service emits custom metrics for model usage:

View metrics: Azure Portal → Application Insights → Metrics → select custom namespace.

Live Metrics

Monitor real-time request rates, failure rates, and dependency durations during the RVAS.

View live metrics: Azure Portal → Application Insights → Live metrics.

Log Analytics (KQL)

Query structured logs across all services:

Copy// End-to-end latency for agent invocations
requests
| where name == "POST /api/agent/invoke"
| summarize avg(duration), percentile(duration, 95) by bin(timestamp, 5m)
| render timechart

Copy// Token usage over time
customMetrics
| where name == "agent.tokens.completion"
| summarize sum(value) by bin(timestamp, 1h)
| render columnchart

Cosmos DB Query Metrics#

Cosmos DB request units (RU) and latency are captured as dependency telemetry in Application Insights. Filter dependencies by type Azure DocumentDB to analyze:

• RU consumption per query
• Latency distribution (p50, p95, p99)
• Throttled requests (HTTP 429)

APIM Analytics Dashboard#

API Management provides built-in analytics:

• Requests: Total calls, success/failure rates, response times by API and operation
• Geography: Request distribution by region
• Errors: Top failing operations with error details
• Capacity: Gateway utilization and backend health

View analytics: Azure Portal → API Management → Analytics.

Infrastructure#

The infrastructure is defined in Bicep modules under infra/:

Contributing#

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/my-feature
3. Make your changes and add tests where applicable
4. Ensure all existing tests pass
5. Commit with a descriptive message: git commit -m "feat: add conversation export"
6. Push to your fork: git push origin feature/my-feature
7. Open a Pull Request against main

Please follow the Conventional Commits specification for commit messages.

License#

This project is licensed under the MIT License.

CopyMIT License

Copyright (c) 2025 AgentOps Control Tower Contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.