Deploy a Leak Detection Scenario on full-multi-node-cluster

This guide walks through a vision-based leak detection scenario built on top of the full-multi-node-cluster blueprint. There is no dedicated leak-detection blueprint; the scenario is enabled by applying the full-multi-node-cluster Terraform with the provided leak-detection.tfvars.example and supporting CI/CD scripts.

The pipeline captures camera frames at the edge, runs AI inference for leak detection, routes alerts to Microsoft Teams, and stores video clips for review.

Total time: ~2 hours (including infrastructure provisioning)

Overview

Architecture

graph LR
    CAM[Camera / RTSP Source]
    MC[508-media-connector]
    INF[507-ai-inference]
    MQTT((MQTT Broker))
    DF[130-messaging Dataflows]
    EH[Event Hub]
    FN[Azure Functions]
    NOTIFY[045-notification → Teams]
    CAP[503-media-capture]
    VQ[520-video-query-api]
    BLOB[(Blob Storage)]

    CAM -->|RTSP frames| MC
    MC -->|Frames via MQTT| MQTT
    MQTT -->|Inference input| INF
    INF -->|ALERT events| MQTT
    MQTT -->|Dataflow routing| DF
    DF -->|Alert events| EH
    EH --> FN
    EH --> NOTIFY
    NOTIFY -->|Teams message| TEAMS[Microsoft Teams]
    INF -->|Capture trigger| CAP
    CAP -->|Video clips| BLOB
    BLOB -->|Query API| VQ

Component Map

Component	Name	Role in Pipeline
508	media-connector	Captures RTSP/ONVIF camera frames, publishes to MQTT
507	ai-inference	Runs ONNX leak detection model on frames, emits ALERT events
503	media-capture	Records video clips to blob storage on alert trigger
509	sse-connector	Server-Sent Events connector for real-time UI streaming
130	messaging	Dataflows routing ALERT events from MQTT to Event Hub
045	notification	Logic App deduplicating alerts and posting to Teams
040	messaging (cloud)	Event Hub and Event Grid for cloud-side event processing
520	video-query-api	REST API for querying stored video captures

Data Flow

Camera Ingestion — 508-media-connector captures frames from ONVIF/RTSP cameras via Akri connectors and publishes them to the MQTT broker
AI Inference — 507-ai-inference consumes frames, runs the ONNX leak detection model, and publishes ALERT events back to MQTT
Edge Routing — 130-messaging dataflows route ALERT events from MQTT to Event Hub
Cloud Processing — Azure Functions process events; 045-notification deduplicates and posts to Teams
Video Capture — 503-media-capture stores video clips to blob storage for later review via 520-video-query-api

Prerequisites

Azure subscription with Contributor access
Azure CLI authenticated (az login)
Terraform >= 1.9.8
Docker installed and running
kubectl configured for your cluster
jq installed for JSON processing
Basic understanding of Azure IoT Operations — see the General User Guide for orientation

Phase 1: Deploy Infrastructure

Estimated time: ~20 minutes + provisioning

The blueprints/full-multi-node-cluster/terraform/ directory contains the infrastructure-as-code for this scenario. A dedicated variable file leak-detection.tfvars.example enables the leak-detection-specific components.

Configure Variables

source scripts/az-sub-init.sh
cd blueprints/full-multi-node-cluster/terraform
cp leak-detection.tfvars.example leak-detection.tfvars

Edit leak-detection.tfvars with your environment values. Key variables to set:

environment — Deployment environment name (e.g., dev)
resource_prefix — Prefix for all resource names (e.g., leakdet)
location — Azure region (e.g., westus3)
instance — Instance identifier (e.g., 001)
teams_recipient_id — Your Teams chat or channel thread ID for alert notifications

Deploy

terraform init
terraform apply -var-file=leak-detection.tfvars

Verify Outputs

After deployment completes, verify the key resources:

terraform output deployment_summary

Confirm the following resources are provisioned:

Resource group
Virtual network and subnets
Key Vault and managed identities
Storage account and Schema Registry
Event Hub namespace with alert Event Hub
Container Registry
VM host with K3s cluster connected to Arc
IoT Operations instance with assets and dataflows

Phase 2: Build and Push Application Images

Estimated time: ~30 minutes

The scenario uses three application container images, built from this repository and pushed to the Azure Container Registry created in Phase 1. The Azure IoT Operations Media Connector is not built here; it is a first-party AIO component enabled by the blueprint.

Image	Source path	Role
`ai-edge-inference`	`src/500-application/507-ai-inference/services/ai-edge-inference`	Rust ONNX/Candle vision inference service that publishes alert events to the AIO MQTT broker.
`sse-server`	`src/500-application/509-sse-connector/services/sse-server`	Python analytics-camera simulator that emits Server-Sent Events consumed by the Akri SSE HTTP connector.
`media-capture-service`	`src/500-application/503-media-capture-service/services/media-capture-service`	Rust workload that maintains an RTSP ring buffer and extracts clips on alert to Azure Blob via ACSA.

Option A: Automated Build

cd blueprints/full-multi-node-cluster

../../src/501-ci-cd/scripts/build-leak-detection-images.sh \
  --acr-name "$(cd terraform && terraform output -raw container_registry | jq -r .name)" \
  --resource-group "$(cd terraform && terraform output -raw deployment_summary | jq -r .resource_group)"

Option B: Manual Build

Build and push the three images individually. The image names must match those produced by the automated build because Phase 3 manifests reference them by these exact tags.

ACR_NAME=$(cd blueprints/full-multi-node-cluster/terraform && terraform output -raw container_registry | jq -r .name)
TAG="latest"

az acr login --name "$ACR_NAME"

# ai-edge-inference (uses Dockerfile.acr, amd64)
docker build \
  -t "$ACR_NAME.azurecr.io/ai-edge-inference:$TAG" \
  -f src/500-application/507-ai-inference/services/ai-edge-inference/Dockerfile.acr \
  src/500-application/507-ai-inference/services/ai-edge-inference
docker push "$ACR_NAME.azurecr.io/ai-edge-inference:$TAG"

# sse-server
docker build \
  -t "$ACR_NAME.azurecr.io/sse-server:$TAG" \
  src/500-application/509-sse-connector/services/sse-server
docker push "$ACR_NAME.azurecr.io/sse-server:$TAG"

# media-capture-service
docker build \
  -t "$ACR_NAME.azurecr.io/media-capture-service:$TAG" \
  src/500-application/503-media-capture-service/services/media-capture-service
docker push "$ACR_NAME.azurecr.io/media-capture-service:$TAG"

For the ai-edge-inference image you can also offload the build to ACR Tasks (avoids needing the local Rust/ONNX toolchain):

az acr build \
  --registry "$ACR_NAME" \
  --image "ai-edge-inference:$TAG" \
  --file src/500-application/507-ai-inference/services/ai-edge-inference/Dockerfile.acr \
  src/500-application/507-ai-inference/services/ai-edge-inference

Verify Images

az acr repository list --name "$ACR_NAME" --output table

Expected repositories: ai-edge-inference, sse-server, media-capture-service.

Phase 3: Deploy Kubernetes Workloads

Estimated time: ~15 minutes

Option A: Automated Deployment

cd blueprints/full-multi-node-cluster

../../src/501-ci-cd/scripts/deploy-leak-detection-apps.sh

Option B: Manual Deployment

Apply manifests in dependency order:

kubectl apply -f ../../src/500-application/508-media-connector/kubernetes/
kubectl apply -f ../../src/500-application/507-ai-inference/kubernetes/
kubectl apply -f ../../src/500-application/503-media-capture/kubernetes/
kubectl apply -f ../../src/500-application/509-sse-connector/kubernetes/

Verify Pods

kubectl get pods -n azure-iot-operations

All application pods should reach Running status.

Phase 4: Configure IoT Operations

Estimated time: ~15 minutes

Camera Asset Definitions

Camera assets are configured through the 111-assets component deployed in Phase 1. Verify the asset definitions:

kubectl get assets -n azure-iot-operations

MQTT Topic Routing

Verify MQTT topics are configured for the inference pipeline:

Input topic: frames from 508-media-connector
Output topic: ALERT events from 507-ai-inference
Dataflow routing: ALERT events forwarded to Event Hub

Dataflow Verification

Confirm the dataflow resources are active:

kubectl get dataflows -n azure-iot-operations

Phase 5: Validate End-to-End

Estimated time: ~10 minutes

Test Event Flow

Verify camera frames are being captured:

kubectl logs -n azure-iot-operations -l app=media-connector --tail=20

Verify inference is processing frames:

kubectl logs -n azure-iot-operations -l app=ai-inference --tail=20

Check Notifications

Trigger a test event and verify the alert appears in the configured Teams channel. The 045-notification Logic App deduplicates alerts by camera_id before posting.

Query Stored Video

After a capture event:

curl -s "https://<video-query-api-url>/api/captures?camera_id=<camera-id>" | jq

Replace <video-query-api-url> and <camera-id> with values from your deployment.

Troubleshooting

ACR authentication failures — Run az acr login --name <acr-name> and verify the managed identity has AcrPull role on the cluster
MQTT topic mismatches — Check the asset definitions in 111-assets match the topic names expected by 507-ai-inference and 508-media-connector
kubectl context — Ensure kubectl config current-context points to your Arc-connected K3s cluster
Notification webhook not firing — Verify teams_recipient_id in terraform.tfvars is a valid Teams chat or channel thread ID
Pods in CrashLoopBackOff — Check container image names match the ACR repository names; verify image pull secrets are configured
No alert events in Event Hub — Confirm the 130-messaging dataflows are active and the MQTT topics are correct

Known Limitations

The 507-ai-inference component ships with a placeholder ONNX model (~0.001 MB). Real leak detection requires a trained industrial safety model.
Container image builds are local-only. CI/CD automation for image builds is a follow-on item.
The blueprint assumes a single-node K3s cluster. Multi-node deployments require the full-multi-node-cluster blueprint as a base.

Next Steps

Customize the inference model — Replace the placeholder ONNX model in 507-ai-inference with a trained leak detection model
Add camera sources — Extend 111-assets definitions to include additional ONVIF/RTSP cameras
Scale to multi-node — Use the full-multi-node-cluster blueprint as a base, then layer leak detection components
Explore other getting-started guides — Browse the Getting Started index for additional walkthroughs and learning resources