Task 04 - Deploy an observability workbook

Introduction

Azure Workbooks help you visualize operational diagnostics for your Azure AI Services resource.

Description

In this task, you will create and deploy an Azure Workbook that visualizes operational diagnostics for your Azure AI Services resource. The workbook uses only platform diagnostics emitted to AzureDiagnostics, without any application-level telemetry.

You will first validate each KQL query manually in Log Analytics, then use GitHub Copilot to generate the Bicep scaffolding and placeholder JSON, and finally paste in the provided workbook definition and deploy it to your environment.

Success Criteria

You have successfully completed Task 04 when:

  1. All three KQL queries return data in Log Analytics when run manually:

    • Request volume
    • Latency percentiles
    • Operation-name breakdown
  2. The workbook deploys successfully using your Bicep template and the provided JSON.
  3. The workbook appears under: Azure Monitor → Workbooks → AI Services Observability
  4. All three visuals load and display real data:
    • Request volume over time
    • Latency percentiles (p50/p95/p99)
    • Requests by operation name over time

Key Tasks

01: Run Diagnostic Queries Manually

Run diagnostic queries in Log Analytics to confirm your Azure AI Services resource is emitting data before creating the workbook.

Expand this section for detailed steps

  1. Navigate to your Log Analytics workspace in the Azure portal.

  2. Open the Logs query editor.

  3. Run the Request Volume query. You should see a time series showing your model invocation volume:

     AzureDiagnostics
 | where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
 | where Category == "RequestResponse"
 | summarize Requests = count() by bin(TimeGenerated, 5m)
 | order by TimeGenerated asc

  4. Run the Latency Percentiles query. You should see p50/p95/p99 latency patterns for all model calls:

     AzureDiagnostics
 | where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
 | where Category == "RequestResponse"
 | summarize
     p50 = percentiles(DurationMs, 50),
     p95 = percentiles(DurationMs, 95),
     p99 = percentiles(DurationMs, 99)
   by bin(TimeGenerated, 5m)
 | order by TimeGenerated asc

  5. Run the Breakdown by operation name query. This shows the number of requests by operation name:

     AzureDiagnostics
 | where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
 | where Category == "RequestResponse"
 | extend Operation = OperationName
 | summarize Count = count() by bin(TimeGenerated, 5m), Operation
 | order by TimeGenerated asc

02: Use Copilot to Create Workbook Scaffolding

Use GitHub Copilot Chat to generate the Bicep template and placeholder JSON file for the Azure Workbook.

Expand this section for detailed steps

  1. Open your project folder in VS Code.

  2. Prompt GitHub Copilot with the following:

     Generate a workbook.bicep template and a placeholder workbook JSON file for an Azure Workbook named "AI Services Observability."
 The workbook will visualize three queries: request volume, latency percentiles, and breakdown by operation name.
 Use a Log Analytics workspace as the data source.
  3. Review the generated workbook.bicep and workbook.json files. Copilot will produce:
    • workbook.bicep
    • workbook.json (placeholder)
  4. If the generated files look correct, approve them.

NOTE Do not provision the workbook yet, proceed to the next step.

03: Replace the Placeholder JSON

Replace the placeholder JSON in the workbook.json file with the final workbook definition that includes all three visualization queries.

Expand this section for detailed steps

  1. Open your project folder in VS Code.

  2. Open the workbook.json file.

  3. Overwrite the file with the following content:

     {
   "version": "Notebook/1.0",
   "items": [
     {
       "type": 1,
       "content": {
         "json": "## AI Services Observability\nThis workbook visualizes Azure AI Services operational data using platform diagnostics from the AzureDiagnostics table.\n\nIt includes:\n- Request volume over time\n- Latency percentiles (p50/p95/p99)\n- Breakdown by operation name"
       },
       "name": "text - 0"
     },
     {
       "type": 3,
       "content": {
         "version": "KqlItem/1.0",
         "query": "AzureDiagnostics\n| where ResourceProvider == \"MICROSOFT.COGNITIVESERVICES\"\n| where Category == \"RequestResponse\"\n| summarize Requests = count() by bin(TimeGenerated, 5m)\n| order by TimeGenerated asc",
         "size": 1,
         "title": "Request Volume Over Time",
         "queryType": 0,
         "resourceType": "microsoft.operationalinsights/workspaces",
         "visualization": "timechart",
         "chartSettings": {
           "yAxis": [
             "Requests"
           ]
         }
       },
       "name": "RequestVolume"
     },
     {
       "type": 3,
       "content": {
         "version": "KqlItem/1.0",
         "query": "AzureDiagnostics\n| where ResourceProvider == \"MICROSOFT.COGNITIVESERVICES\"\n| where Category == \"RequestResponse\"\n| summarize\n p50 = percentiles(DurationMs, 50),\n p95 = percentiles(DurationMs, 95),\n p99 = percentiles(DurationMs, 99)\n by bin(TimeGenerated, 5m)\n| order by TimeGenerated asc",
         "size": 1,
         "title": "Latency Percentiles (p50 / p95 / p99)",
         "queryType": 0,
         "resourceType": "microsoft.operationalinsights/workspaces",
         "visualization": "timechart"
       },
       "name": "LatencyTrends"
     },
     {
       "type": 3,
       "content": {
         "version": "KqlItem/1.0",
         "query": "AzureDiagnostics\n| where ResourceProvider == \"MICROSOFT.COGNITIVESERVICES\"\n| where Category == \"RequestResponse\"\n| extend Operation = OperationName\n| summarize Count = count() by bin(TimeGenerated, 5m), Operation\n| order by TimeGenerated asc",
         "size": 1,
         "title": "Requests by Operation Name Over Time",
         "queryType": 0,
         "resourceType": "microsoft.operationalinsights/workspaces",
         "visualization": "timechart",
         "chartSettings": {
           "yAxis": [
             "Count"
           ]
         }
       },
       "name": "OperationBreakdown"
     }
   ],
   "fallbackResourceIds": [
     "azureResourceId"
   ],
   "$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json"
 }

  4. Save the file.

04: Deploy the Workbook

Deploy the workbook to Azure using Azure Developer CLI to provision the workbook resource.

Expand this section for detailed steps

  1. Run azd provision to deploy the workbook:

     azd provision

  2. Once the deployment is complete, check the Azure portal to confirm the workbook is deployed.

05: Review in the Portal

Navigate to the workbook in the Azure portal and verify that all visuals render correctly and display data matching your manual KQL queries.

Expand this section for detailed steps

  1. Open the Azure portal.

  2. Navigate to Azure Monitor → Workbooks → AI Services Observability.

  3. Confirm that the workbook loads and all visuals render.

  4. Confirm that the data matches the output from your manual KQL queries.

  5. Close the workbook.

Summary

You’ve completed this task. You deployed an Azure Monitor workbook that visualizes operational diagnostics for your Azure AI Services resource. You used Copilot to create the workbook scaffolding and replace the placeholder JSON. You also validated the workbook by running manual KQL queries and reviewing the data in the portal.