Section 15 - Schedule Runbooks

SPARK utilizes runbook schedules to maintain data accuracy and compliance. This ensures SharePoint site inventories are updated in SQL for attestation and permissions as well as keeping DL membership synchronized between Exchange Online and Logic Apps workflows. Without scheduling, stale data can lead to compliance gaps or incorrect access permissions.

It also helps Optimize Resource Utilization. SPARK deployments often involve heavy operations like SQL updates and DL management. Scheduling runbooks during off-peak hours can minimizes CPU and I/O contention along with allowing controlled execution of resource-intensive scripts (e.g., ForceFullRebuild steps).

TODO

This should only be run when the environment is ready to go live.

Reduce Operational Risk

Before scheduling be sure to: 1. Publish runbooks (drafts won’t trigger). 2. Test manually to confirm functionality. 3. Validate permissions and automation account identity.

This prevents failures during unattended execution and ensures reliability.

Schedule

The following runbooks will need to be scheduled when your environment is ready to go live. You have the freedom to schedule said runbooks to best suit your needs.

Purpose	Runbook	Schedule Name	Example	Paramaters
Distribution list notifications	rb-spark-manage-dl-notifications	rb-spark-manage-dl-notifications daily at * ****	rb-spark-manage-dl-notifications daily at 7pm	“ForceFullRebuild” = “False”; “ShowLogs” = “False”
Site collector	rb-spark-sitecollector	rb-spark-sitecollector daily at * ****	rb-spark-sitecollector daily at 9pm	“NewSitesOnly” = “True”; “ShowLogs” = “False”
Updated the error action to match for the loading of the spark libraries	rb-spark-changed-site-mgmt	rb-spark-changed-site-mgmt daily at * ****	rb-spark-changed-site-mgmt daily at 11pm	“ShowLogs” = “False”; SiteUrls leave blank (optional); EvaluateSitesModifiedAfter leave blank (optional); SiteCountToReturn leave blank (optional) defaults to 2000
Site inventory management	rb-spark-siteinventory-mgmt	Site Inventory Management daily at * ****	rb-spark-siteinventory-mgmt daily at 2am	“ShowLogs” = “False”
Manage site crawl state	rb-spark-manage-crawl-state	rb-spark-manage-crawl-state daily at * ****	rb-spark-manage-crawl-state daily at 4am	“ShowLogs” = “False”
Email dispatch to DLs	rb-spark-sendemail-toDL	rb-spark-sendemail-toDL daily at * ****	rb-spark-sendemail-toDL daily at 7am	None
Database Maintenance	rb-spark-db-maintenance	rb-spark-db-maintenance daily at * ****	rb-spark-db-maintenance daily at 5am	None

PowerShell Script

Review the script and install the required modules if needed
Update the Azure Automation Account and Schedule information
- ResourceGroup - The Resource Group associated with your Automation Account
- AutomationAcct - The Automation Account with the runbooks you want to schedule
- TimeZoneId - The Time Zone to base the schedule Date Time on. I.E. Eastern Standard Time
For each Schedule update the following information
- ScheduleName - The schedule name. I.E. rb-spark-manage-crawl-state daily at 1 am
- Hour - The hour in military time to start the runbook. I.E. 23 for 11pm
- Minute - The minute to start the runbook. I.E. 0 to 59
Review the permissions and azure roles to apply to the uami
Run the script and validate the schedules are created, linked and disabled
During the validation phase, please enable the schedules associated with:
- rb-spark-sitecollector
- rb-spark-siteinventory-mgmt
- rb-spark-manage-dl-notifications
- rb-spark-changed-site-mgmt
Once you are ready to go live, please enable the schedules associated with:
- rb-spark-sendemail-toDL
- rb-spark-manage-crawl-state

########################################## Required Modules ##########################################
#Install-Module Az.Az.Accounts
#Install-Module Az.Automation

########################################## Import Modules
##########################################
# Load required modules
Import-Module Az.Accounts
Import-Module Az.Automation

########################################## Connect to Azure ##########################################
# Connect to Azure
# Connect to Azure with the appropriate environment
#You need to have previously connected to Azure

$AzContext = Get-AzContext

[void](Get-AzDefault -ErrorVariable azdefault -ErrorAction SilentlyContinue)

if($AzContext -and [string]$azdefault -match 'Run Connect-AzAccount to login.' ){

    Write-host "Found connection to Az with the following information"  

    Write-host -ForegroundColor DarkCyan "Account Id: " -NoNewline; Write-Host $($AzContext.Account.Id)

    Write-host -ForegroundColor DarkCyan "Subscription Name: " -NoNewline; Write-Host $($AzContext.Subscription.Name)

    Write-Host -ForegroundColor DarkCyan "Subscription Id: "  -NoNewline; Write-Host $($AzContext.Subscription.id)

    Write-Host -ForegroundColor DarkCyan "Azure Environment: "; Write-Host $AzContext.Environment

    $answer = Read-Host "Is this the correct Account to use? (Y/N)"

}

else{

    $answer = 'N'

}

if($Answer -eq 'Y'){

    Connect-AzAccount -AccountId $AzContext.Account.Id -Subscription $AzContext.Subscription.Id -Environment $AzContext.Environment

}

else{

    Write-host -ForegroundColor Yellow "Please connect to Azure"
    
    $envChoice = Read-Host "Enter environment (AzureCloud/AzureUSGovernment/AzureChinaCloud)"
    Connect-AzAccount -Environment $envChoice

    
}

########################################## Global Inputs
##########################################
# --- Global Inputs ---
$ResourceGroup   = "dau-rg-spark-fa-test"
$AutomationAcct  = "aa-spark-automation0"
$TimeZoneId      = "Eastern Standard Time"  # Handles DST, Other Time zones: Pacific Standard Time, Central Standard Time, UTC, GMT Standard Time, Mountain Standard Time, 	Hawaiian Standard Time

########################################## Runbook Configuration
##########################################
# Each runbook can have unique parameters and run settings rb-spark-sitecollector
# For each runbook schedule, you will need to update the Schedule name, hour and minute
$RunbookSchedules = @(
    @{
        RunbookName    = "rb-spark-sitecollector"
        ScheduleName   = "rb-spark-sitecollector daily at 10 pm" #update name to reflect hour and Minute
        Hour           = 22 #update
        Minute         = 0 #update
        Parameters     = @{ "NewSitesOnly" = "True"; "ShowLogs" = "False" }
    },
    @{
        RunbookName    = "rb-spark-siteinventory-mgmt"
        ScheduleName   = "rb-spark-siteinventory-mgmt daily at 11 pm" #update name to reflect hour and Minute
        Hour           = 23 #update
        Minute         = 0 #update
        Parameters     = @{ "ShowLogs" = "False" }
    },
    @{
        RunbookName    = "rb-spark-manage-crawl-state"
        ScheduleName   = "rb-spark-manage-crawl-state daily at 1 am" #update name to reflect hour and Minute
        Hour           = 1 #update
        Minute         = 0 #update
        Parameters     = @{ "ShowLogs" = "False" }
    }
    @{
        RunbookName    = "rb-spark-manage-dl-notifications"
        ScheduleName   = "rb-spark-manage-dl-notifications daily at 3 pm" #update name to reflect hour and Minute
        Hour           = 15 #update
        Minute         = 0 #update
        Parameters     = @{ "ForceFullRebuild" = "False"; "ShowLogs" = "False" }
    }
    @{
        RunbookName    = "rb-spark-sendemail-toDL"
        ScheduleName   = "rb-spark-sendemail-toDL daily at 7 am" #update name to reflect hour and Minute
        Hour           = 7 #update
        Minute         = 0 #update
        Parameters     = @{  }
    }
     @{
        RunbookName    = "rb-spark-changed-site-mgmt"
        ScheduleName   = "rb-spark-changed-site-mgmt daily at 2 am" #update name to reflect hour and Minute
        Hour           = 2 #update
        Minute         = 0 #update
        Parameters     = @{ "ShowLogs" = "False"  }
    }
)


########################################## Configure functions
##########################################

# --- Function: Compute next occurrence ---
function Get-NextOccurrence($Hour, $Minute, $TimeZoneId) {
    $tz      = [System.TimeZoneInfo]::FindSystemTimeZoneById($TimeZoneId)
    $nowUtc  = [DateTimeOffset]::UtcNow
    $nowEast = [System.TimeZoneInfo]::ConvertTime($nowUtc, $tz)

    $targetTime = if ($nowEast.Hour -lt $Hour -or ($nowEast.Hour -eq $Hour -and $nowEast.Minute -lt $Minute)) {
        $nowEast.Date.AddHours($Hour).AddMinutes($Minute)
    } else {
        $nowEast.Date.AddDays(1).AddHours($Hour).AddMinutes($Minute)
    }

    return [DateTimeOffset]::new($targetTime, $tz.GetUtcOffset($targetTime))
}

# --- Function: Validate parameters ---
function Validate-RunbookParameters($RunbookName, $ProvidedParams) {
    $runbook = Get-AzAutomationRunbook `
        -ResourceGroupName $ResourceGroup `
        -AutomationAccountName $AutomationAcct `
        -Name $RunbookName

    $validParams = $runbook.Parameters.Keys
    $invalidParams = $ProvidedParams.Keys | Where-Object { $_ -notin $validParams }

    return $invalidParams
}

# --- Function: Create schedule and link ---
function Create-ScheduleAndLink($ResourceGroup, $AutomationAcct, $RunbookName, $ScheduleName, $Hour, $Minute, $TimeZoneId, $Parameters, $RunOn) {
    try {
        # Validate parameters
        $invalidParams = Validate-RunbookParameters -RunbookName $RunbookName -ProvidedParams $Parameters
        if ($invalidParams.Count -gt 0) {
            Write-Error "Invalid parameters for runbook '$RunbookName': $($invalidParams -join ', ')"
            return
        }

        # Check if schedule already exists
        $existingSchedule = Get-AzAutomationSchedule `
            -ResourceGroupName $ResourceGroup `
            -AutomationAccountName $AutomationAcct `
            -Name $ScheduleName `
            -ErrorAction SilentlyContinue

        if ($existingSchedule) {
            Write-Host "Schedule '$ScheduleName' already exists. Skipping creation."
        }
        else {
            $startDto = Get-NextOccurrence -Hour $Hour -Minute $Minute -TimeZoneId $TimeZoneId

            # Create schedule
            New-AzAutomationSchedule `
                -ResourceGroupName $ResourceGroup `
                -AutomationAccountName $AutomationAcct `
                -Name $ScheduleName `
                -Description "Runs daily at $Hour : $Minute Eastern" `
                -StartTime $startDto `
                -TimeZone $TimeZoneId `
                -DayInterval 1

            Write-Host "Created schedule '$ScheduleName'."
        }

        # Link schedule to runbook (only if not already linked)
        $linkedSchedules = Get-AzAutomationScheduledRunbook `
            -ResourceGroupName $ResourceGroup `
            -AutomationAccountName $AutomationAcct `
            -RunbookName $RunbookName `
            -ErrorAction SilentlyContinue

        if ($linkedSchedules.ScheduleName -contains $ScheduleName) {
            Write-Host "Runbook '$RunbookName' is already linked to schedule '$ScheduleName'."
        }
        else {
            Register-AzAutomationScheduledRunbook `
                -ResourceGroupName $ResourceGroup `
                -AutomationAccountName $AutomationAcct `
                -RunbookName $RunbookName `
                -ScheduleName $ScheduleName `
                -Parameters $Parameters

            Write-Host "Linked runbook '$RunbookName' to schedule '$ScheduleName'."
        }
    }
    catch {
        Write-Error "Failed to create or link schedule '$ScheduleName'. Details: $_"
    }
}

# --- Function: Disable schedule ---
function Disable-Schedule($ResourceGroup, $AutomationAcct, $ScheduleName) {
    try {
        Set-AzAutomationSchedule `
            -ResourceGroupName $ResourceGroup `
            -AutomationAccountName $AutomationAcct `
            -Name $ScheduleName `
            -IsEnabled $false

        Write-Host "Disabled schedule '$ScheduleName'."
    }
    catch {
        Write-Error "Failed to disable schedule '$ScheduleName'. Details: $_"
    }
}
########################################## Main Execution ##########################################
# Create and Link Schedule to associated Runbook
foreach ($item in $RunbookSchedules) {
    Create-ScheduleAndLink `
        -ResourceGroup $ResourceGroup `
        -AutomationAcct $AutomationAcct `
        -RunbookName $item.RunbookName `
        -ScheduleName $item.ScheduleName `
        -Hour $item.Hour `
        -Minute $item.Minute `
        -TimeZoneId $TimeZoneId `
        -Parameters $item.Parameters `
        -RunOn $item.RunOn
}

# Disable all schedules after creation
foreach ($item in $RunbookSchedules) {
    Disable-Schedule `
        -ResourceGroup $ResourceGroup `
        -AutomationAcct $AutomationAcct `
        -ScheduleName $item.ScheduleName
}

########################################## Disconnect ##########################################
Disconnect-AzAccount | Out-Null;

Manual Steps:

Step 1: Create Schedule

Browse and log into the Azure Portal

Use the correct URL for your environment:

Worldwide (Commercial) & GCC
https://portal.azure.com

GCC-High and DoD
https://portal.azure.us

From the top search bar, enter Automation Account and select it from the search results

Click on aa-spark-automation0

On the left hand side under Shared Resources, click schedule and then click Add a schedule. Set the properties shown in the table and click Create.

Name	Value
Name	Site Inventory Management Daily 10pm
Description	Runs rb-spark-siteinventory-mgmt everyday at 10 pm to ensure new sites are inventoried properly
Start Date	Enter Date i.e. today’s date
Start Time	Enter Desired Time i.e. 10pm
Time Zone	Enter appropriate Time Zone i.e. Eastern
Recurrence	Recurring
Recur Every	1 day
Set Expiration	No

Step 2: Attach Schedule to associated Runbook

Within the aa-spark-automation0 Automation Account, click Process Automation on the left hand side and select Runbooks. Select rb-spark-siteinventory-mgmt.

Select rb-spark-siteinventory-mgmt runbook

On the left hand side under Resources, select Schedules. Click Add a Schedule.

In the next window, click on Schedule to associate the appropriate schedule.

Select the appropriate schedule (i.e. rb-spark-siteinventory-mgmt everyday at 11 pm ).

Next select “Parameters and run settings”. Set the properties shown in the table and click Ok.

Name	Value
ShowLogs	False
Run on	Azure

Click Ok again in the Schedule Runbook window to save the schedule.

You may need to click on refresh to view the attached schedule.

Repeat the above steps with the appropriate scheduled and date time for the remaining runbooks.