Implementation | Session 13: Regional failover rehearsal for a governed AI service

Chapter 5 of 6

Implement#

1. Complete the rehearsal contract#

Set the approved scope, customer-control paths, selectors, expected values, and runbook owners. Then check the local contract.

PowerShell

.\scripts\preflight.ps1 `
  -Phase Decisions `
  -ApprovedScope "/subscriptions/<id>/resourceGroups/<name>"

Bash

./scripts/preflight.sh \
  --phase decisions \
  --approved-scope "/subscriptions/<id>/resourceGroups/<name>"

2. Check the active primary path#

Use the external runtime directory for temporary health output. Ready preflight checks the approved primary path before the team previews a move.

PowerShell

.\scripts\preflight.ps1 `
  -Phase Ready `
  -ApprovedScope "/subscriptions/<id>/resourceGroups/<name>" `
  -RuntimeDirectory "C:\customer-runtime"

Bash

./scripts/preflight.sh \
  --phase ready \
  --approved-scope "/subscriptions/<id>/resourceGroups/<name>" \
  --runtime-directory "/customer/runtime"

3. Rehearse failover and restore#

Freeze infrastructure and API Management policy changes. Review the move preview. Get delivery-owner approval before starting. The wrapper checks secondary readiness, moves the selector, checks the secondary path, previews the return, restores the primary selector, and checks the primary path.

PowerShell

.\scripts\rehearse-failover.ps1 `
  -ApprovedScope "/subscriptions/<id>/resourceGroups/<name>" `
  -ChangeRecordId "<customer-change-record>" `
  -RuntimeDirectory "C:\customer-runtime" `
  -Confirm

Bash

./scripts/rehearse-failover.sh \
  --approved-scope "/subscriptions/<id>/resourceGroups/<name>" \
  --change-record-id "<customer-change-record>" \
  --runtime-directory "/customer/runtime" \
  --confirm