Agent Workload — Reference Component

Part of the Frontier Fabric AgentOps RVAS. This is the telemetry source you deploy in Challenge 1 — a full-stack Azure AI Foundry agent that emits backend-origin traces, custom token and latency metrics, and conversation data the Control Tower later correlates.

Full-stack AI agent runtime with application observability from the backend to model inference.

Architecture Overview#

The Agents Runtime is the core execution block of the observability platform. It processes requests across three microservices and persists conversation data. Application telemetry begins at the backend; the frontend is not instrumented in this reference workload.

Architecture Diagram#

CopyUser → Frontend (Next.js) → Backend (FastAPI) → Agent (FastAPI) → Azure AI Foundry
                │
                └→ Cosmos DB (agentsdb)

Backend + Agent → Application Insights

API Management is provisioned for future gateway integration but is not in this active path.

Prerequisites#

Quick Start#

Copy# Authenticate with Azure
azd auth login

# Initialize the environment (first time only)
azd init

# Provision infrastructure and deploy all services
azd up

Docker must be running. azd up is the normal path when containers can download packages from npm and PyPI. If a restricted network blocks local container package downloads, run azd provision, build the agent, backend, and frontend images as latest with az acr build, and attach them with new Container Apps revisions. See the Challenge 1 coach guide for the exact fallback commands.

After deployment completes, azd prints the service URLs:

Copy  frontend  https://frontend.<env>.<region>.azurecontainerapps.io
  backend   https://backend.<env>.<region>.azurecontainerapps.io
  agent     https://agent.<env>.<region>.azurecontainerapps.io

Run azd env get-values and record the Application Insights, Log Analytics, and Cosmos DB coordinates for the following challenge.

Services#

Frontend — src/frontend#

Backend — src/backend#

Agent — src/agent#

API Reference#

Backend Service (/api)#

Create a Conversation

CopyPOST /api/conversations
Content-Type: application/json

{
  "title": "New Conversation"
}

Response 201 Created

Copy{
  "id": "conv-abc123",
  "title": "New Conversation",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Get a Conversation

CopyGET /api/conversations/{id}

Response 200 OK

Copy{
  "id": "conv-abc123",
  "title": "New Conversation",
  "messages": [
    {
      "role": "user",
      "content": "Hello!",
      "timestamp": "2024-01-15T10:31:00Z"
    },
    {
      "role": "assistant",
      "content": "Hi there! How can I help you today?",
      "timestamp": "2024-01-15T10:31:02Z"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:31:02Z"
}

Send a Message

CopyPOST /api/conversations/{id}/messages
Content-Type: application/json

{
  "role": "user",
  "content": "Explain observability in distributed systems."
}

Response 200 OK

Copy{
  "userMessage": {
    "id": "message-user-123",
    "conversationId": "conv-abc123",
    "sessionId": "session-abc123",
    "role": "user",
    "content": "Explain observability in distributed systems.",
    "timestamp": "2024-01-15T10:32:01Z",
    "status": "accepted"
  },
  "assistantMessage": {
    "id": "message-assistant-456",
    "conversationId": "conv-abc123",
    "sessionId": "session-abc123",
    "role": "assistant",
    "content": "Observability in distributed systems refers to...",
    "timestamp": "2024-01-15T10:32:05Z",
    "model": "gpt-5-mini",
    "promptTokens": 128,
    "completionTokens": 256,
    "totalTokens": 384,
    "durationMs": 1834.7,
    "status": "succeeded"
  }
}

Both message documents are persisted to the Cosmos DB interactions container. The assistant document carries the model-usage fields consumed by Fabric Mirroring and downstream agent analytics.

Health Check (Backend)

CopyGET /api/health

Response 200 OK

Copy{ "status": "healthy", "service": "backend", "version": "1.0.0" }

Agent Service (/api/agent)#

Invoke Agent (Synchronous)

CopyPOST /api/agent/invoke
Content-Type: application/json

{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is Azure Container Apps?" }
  ],
  "temperature": 0.7,
  "max_tokens": 1024
}

Response 200 OK

Copy{
  "response": "Azure Container Apps is a serverless container platform...",
  "usage": {
    "prompt_tokens": 64,
    "completion_tokens": 200
  },
  "model": "gpt-5-mini",
  "duration_ms": 1412.6
}

Invoke Agent (Streaming)

CopyPOST /api/agent/stream
Content-Type: application/json

{
  "messages": [
    { "role": "user", "content": "Write a haiku about cloud computing." }
  ]
}

Response 200 OK (Server-Sent Events)

Copydata: {"delta": "Servers ", "type": "content"}
data: {"delta": "in the sky,", "type": "content"}
data: {"delta": "\n", "type": "content"}
data: {"delta": "Scaling ", "type": "content"}
data: {"delta": "without limits—", "type": "content"}
data: {"delta": "\n", "type": "content"}
data: {"delta": "Code ", "type": "content"}
data: {"delta": "runs everywhere.", "type": "content"}
data: {"type": "done", "usage": {"prompt_tokens": 32, "completion_tokens": 18, "total_tokens": 50}}

Health Check (Agent)

CopyGET /api/health

Response 200 OK

Copy{ "status": "healthy", "service": "agent" }

Environment Variables#

Backend Service#

Agent Service#

Frontend Service#

Local Development#

1. Backend#

Copycd src/backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Set required environment variables
export COSMOS_DB_ENDPOINT="https://<your-account>.documents.azure.com:443/"
export COSMOS_DATABASE="agentsdb"
export AGENT_SERVICE_URL="http://localhost:8001"
export APPLICATIONINSIGHTS_CONNECTION_STRING="<your-connection-string>"

uvicorn app:app --reload --port 8000

2. Agent#

Copycd src/agent
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Set required environment variables
export AZURE_OPENAI_ENDPOINT="https://<your-resource>.openai.azure.com/"
export AZURE_OPENAI_DEPLOYMENT="gpt-5-mini"
export APPLICATIONINSIGHTS_CONNECTION_STRING="<your-connection-string>"

uvicorn app:app --reload --port 8001

3. Frontend#

Copycd src/frontend
npm install

# Create .env.local
echo 'NEXT_PUBLIC_API_URL=http://localhost:8000' > .env.local

npm run dev

The frontend is available at http://localhost:3000.

Monitoring & Observability#

Application Insights Integration#

Application telemetry begins at the backend. The backend and agent use the Azure Monitor OpenTelemetry SDK for inbound requests, outbound HTTP dependencies, Azure SDK dependencies, and custom metrics. The frontend is not instrumented.

Distributed Tracing

Inbound backend requests create trace context that flows through the instrumented service chain:

CopyBackend → Agent → Azure OpenAI
  └────────────→ Cosmos DB

Request and dependency spans include:

• HTTP method, URL, and status code
• Duration and dependency details
• Operation and trace correlation identifiers

View traces: Azure Portal → Application Insights → Transaction search → filter by operation name.

Application Map

The Application Map automatically discovers service dependencies and displays:

• Request rates and failure rates between services
• Average response times per dependency
• Cosmos DB and Azure OpenAI as external dependencies

View the map: Azure Portal → Application Insights → Application map.

Custom Metrics

The agent service emits custom metrics for model usage:

View metrics: Azure Portal → Application Insights → Metrics → select custom namespace.

Live Metrics

Monitor real-time request rates, failure rates, and dependency durations during the RVAS.

View live metrics: Azure Portal → Application Insights → Live metrics.

Log Analytics (KQL)

Confirm named backend and agent requests:

CopyAppRequests
| where TimeGenerated > ago(30m)
| project TimeGenerated, AppRoleName, Name, OperationId, ResultCode, Success
| order by TimeGenerated desc

Confirm correlated service, model, and Cosmos DB dependencies:

CopyAppDependencies
| where TimeGenerated > ago(30m)
| project TimeGenerated, AppRoleName, Name, Target, OperationId, ResultCode, Success
| order by TimeGenerated desc

Confirm token and latency metrics:

CopyAppMetrics
| where TimeGenerated > ago(30m) and Name startswith "agent."
| summarize Value=sum(Sum) by Name, bin(TimeGenerated, 5m)
| order by TimeGenerated desc

Cosmos DB Query Metrics#

Cosmos DB request units (RU) and latency are captured as dependency telemetry in Application Insights. Filter dependencies by type Azure DocumentDB to analyze:

• RU consumption per query
• Latency distribution (p50, p95, p99)
• Throttled requests (HTTP 429)

APIM Analytics Dashboard#

APIM is provisioned but is not in the reference application's active request path. Its analytics will remain empty until clients are explicitly routed through the APIM gateway.

API Management provides built-in analytics:

• Requests: Total calls, success/failure rates, response times by API and operation
• Geography: Request distribution by region
• Errors: Top failing operations with error details
• Capacity: Gateway utilization and backend health

View analytics: Azure Portal → API Management → Analytics.

Infrastructure#

The infrastructure is defined in Bicep modules under infra/:

Contributing#

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/my-feature
3. Make your changes and add tests where applicable
4. Ensure all existing tests pass
5. Commit with a descriptive message: git commit -m "feat: add conversation export"
6. Push to your fork: git push origin feature/my-feature
7. Open a Pull Request against main

Please follow the Conventional Commits specification for commit messages.

License#

This project is licensed under the MIT License.

CopyMIT License

Copyright (c) 2025 AgentOps Control Tower Contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.