Section 8 - Create the Automation Accounts

SPARK utilizes Azure Automation Accounts1. By default, five automation accounts are established within the tenant. The first automation account, aa-spark-autoamation0 acts as the primary. All additional automation accounts are secondary and used only for batch and parallel processing time intensive operations.

Requirements

The Azure Administrator will be required for this step. The user must have:

  • Owner role for the subscription

Manual Steps:

Video Walkthrough

Step 1 - Create Azure Runbooks

This section will utilize scripts to create the Azure runbooks. Complete the following steps to run the script.

  1. Download the scripts from the table below and place them in the same folder
Scripts
ProvisionAzureAutomationAccounts
rb-spark-changed-site-mgmt
rb-spark-db-maintenance
rb-spark-lib
rb-spark-manage-crawl-state
rb-spark-manage-dl-notifications
rb-spark-sendemail-toDL
rb-spark-set-sql-notification-state
rb-spark-sitecollector
rb-spark-siteinventory-mgmt
  1. Review the script below
  2. Under the Variables section, set the following values
    • tenantSiteId - The SPO admin center site id from Section 5 Step 6
    • rgName - The resource group name

To get the tenant site id if Section 5 Step 6 wasn’t completed, do the following:

  1. Access the SPO Admin Center (Ex: https://tenant-admin.sharepoint.com)
  2. Update the url to be: https://tenant-admin.sharepoint.com/_api/site/id
  1. Under the Connect to Azure section, uncomment the appropriate connection line for your environment
  2. Run the script to generate the variables.txt file
  3. Copy the values from the output to the ProvisionAzureAutomationAccounts script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210

########################################## Required Modules ##########################################
#Install-Module Az.Accounts -Scope CurrentUser
#Install-Module Microsoft.Graph.Applications -Scope CurrentUser

################################################## Variables ##################################################
# Set the SPO admin center site id and resource group
$tenantSiteId = ''
$rgName = 'sub-spark-rg'

# Static variables for the provision script
$staticVariables = @{
    automationAccountNameBase = "aa-spark-automation"
    crawlEEEU                 = "false"
    exo_cert_name             = "appreg-spark-sendmail-cert"
    lastRunTimeLSWP           = "01/01/2020 21:00:00"
    managedIdentity           = "true"
    permissionRunbooksRunning = "-14"
    retaindeletedsites        = "1"
    RuntimeVersion51          = "5.1"
    RuntimeVersion72          = "7.2"
    showLogs                  = "0"
    siteCollectorContinued    = "False"
    siteCollectorRunning      = "0"
    uami                      = "true"
}

# Variables from the environment fo the provision script
$variables = @{
    azAccountEnvironment = ""
    azureEnvironment     = ""
    catchAllAccount      = ""
    clientId             = ""
    exo_app_id           = ""
    exo_email_suffix     = "@"
    exo_environment      = ""
    exo_organization     = ""
    exo_sendEmailApp     = ""
    exo_sendEmailNPE     = "org-spark-mail@"
    graphUrl             = ""
    helpurl              = ""
    location             = ""
    MgGraph_Environment  = ""
    resourceGroupName    = ""
    siteadminurl         = ""
    sporootURL           = ""
    sqlConnectionString  = ""
    subscriptionId       = ""
    subscriptionName     = ""
    tenant               = ""
    tenantId             = ""
    tenantSiteId         = $tenantSiteId
    tenantUrl            = ""
    uamiName             = "uami-spark-spoactions"
}

################################################## Connect to Azure ##################################################
#Import-Module Az.Accounts

# Log
Write-Host "Connecting to Azure..."

# Connect to Azure
# Commercial/GCC
#Connect-AzAccount | Out-Null;

# GCC-H/DoD
#Connect-AzAccount -Environment AzureUSGovernment | Out-Null;

################################################## Environment ##################################################
# Log
Write-Host "Getting the Azure context information..."

# Get the context of Azure
$azContext = Get-AzContext;

# Log
Write-Host "Getting the tenant information..."

# Get the tenant information
$tenantDetails = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$($azContext.Tenant.Id)/.well-known/openid-configuration" -Method Get

# Set the variables, based on the graph endpoint
switch ($tenantDetails.msgraph_host) {
    'graph.microsoft.com' {
        $envType = "Default"
        $spoDomain = "sharepoint.com";
        $variables.exo_environment = "O365DEFAULT";
        $variables.graphUrl = "https://$($tenantDetails.msgraph_host)";
        $variables.MgGraph_Environment = "Global";
        $variables.sqlConnectionString = "Server=tcp:sqldb-spark.database.windows.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;"
    }
    'graph.microsoft.us' {
        $envType = "GCC-H"
        $spoDomain = "sharepoint.us";
        $variables.exo_environment = "O365USGovGCCHigh";
        $variables.graphUrl = "https://$($tenantDetails.msgraph_host)";
        $variables.MgGraph_Environment = "USGov";
        $variables.sqlConnectionString = "Server=tcp:sqldb-spark.database.usgovcloudapi.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;"
    }
    'dod-graph.microsoft.us' {
        $envType = "DoD"
        $spoDomain = "sharepoint-mil.us";
        $variables.exo_environment = "O365USGovDoD";
        $variables.graphUrl = "https://$($tenantDetails.msgraph_host)";
        $variables.MgGraph_Environment = "USGovDoD";
        $variables.sqlConnectionString = "Server=tcp:sqldb-spark.database.usgovcloudapi.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;"
    }
}

################################################## Azure ##################################################
# Set the variables
$variables.azAccountEnvironment = $azContext.Environment.Name;
$variables.azureEnvironment = $azContext.Environment.Name;
$variables.exo_email_suffix += $azContext.Tenant.DefaultDomain;
$variables.exo_organization = $azContext.Tenant.DefaultDomain;
$variables.exo_sendEmailNPE += $azContext.Tenant.DefaultDomain;
$variables.subscriptionId = $azContext.Subscription.Id;
$variables.subscriptionName = $azContext.Subscription.Name;
$variables.tenant = $azContext.Tenant.DefaultDomain;
$variables.tenantId = $azContext.Tenant.Id;

# Determine the sharepoint urls
$domain = $variables.tenant.Split(".")[0];
$variables.helpurl = "https://$domain.$spoDomain/sites/spark/sitepages/attestation-portal-help.aspx";
$variables.siteadminurl = "https://$domain-admin.$spoDomain";
$variables.sporootURL = "https://$domain.$spoDomain";
$variables.tenantUrl = $variables.siteadminurl;

# Set the variables
$variables.catchAllAccount = "org-spark-mail@" + $variables.tenant;

# Log
Write-Host "Getting the resource group $rgName information..."

# Get the resource group
$rg = Get-AzResourceGroup -Name $rgName
if($rg -ne $null) {
    # Set the variables
    $variables.location = $rg.Location;
    $variables.resourceGroupName = $rg.ResourceGroupName;
} else {
    # Log
    Write-Host "Resource group $rgName was not found"
}

# Log
Write-Host "Getting the UAMI information..."

# Get the UAMI
$uami = Get-AzUserAssignedIdentity -ResourceGroupName $rgName -Name uami-spark-spoactions;
if($uami -ne $null) {
    # Set the variables
    $variables.clientId = $uami.ClientId;
} else {
    # Log
    Write-Host "UAMI was not found"
}

# Disconnect from Azure
Disconnect-AzAccount | Out-Null;

################################################## Graph ##################################################
# Required Module:
Import-Module Microsoft.Graph.Applications;

# Connect to Graph
Write-Host "Connecting to Graph...";
if ($envType -eq 'GCC-H') {
    # GCC High
    Connect-MgGraph -Environment UsGov -Scopes "Application.Read.All"
} else {
    if ($envType -eq 'DoD') {
        # DoD
        Connect-MgGraph -Environment UsGovDoD -Scopes "Application.Read.All"
    }
    else {
        # Commercial/GCC
        Connect-MgGraph -Scopes "Application.Read.All" | Out-Null;
    }
}

# Log
Write-Host "Getting the sendmail app registration..."

# Get the application
$app = Get-MgApplication -Filter "displayName eq 'appreg-spark-sendmail'";
if($app -ne $null) {
    # Set the variables
    $variables.exo_app_id = $app.AppId;
    $variables.exo_sendEmailApp = $app.AppId;
} else {
    # Log
    Write-Host "appreg-spark-sendmail app registration was not found"
}

# Disconnect from Graph
Disconnect-MgGraph | Out-Null;

################################################## Output ##################################################
# Output the variables
"# Variables that need to be updated from the spreadsheet" | Out-File -FilePath "./variables.txt";
$variables.Keys | Sort-Object | foreach-object {
    ("`$$_ = `"" + $variables[$_] + "`";") | Out-File -Append -FilePath "./variables.txt";
}

# Output the static variables
"`n# Static variables - Do not change" | Out-File -Append -FilePath "./variables.txt";
$staticVariables.Keys | Sort-Object | foreach-object {
    ("`$$_ = `"" + $staticVariables[$_] + "`";") | Out-File -Append -FilePath "./variables.txt";
}
  1. Reference the variables spreadsheet and update the ProvisionAzureAutomationAccounts.ps1 script variables found in the Variable section
  2. Open PowerShell v7.2+
  3. Run the script and validate the runbooks are created

Create Runbook Error: “Conflict”

When running this automation script, you may run into an error:

New-AzAutomationAccount: Operation returned an invalid status code ‘Conflict’

This is based on the limitation of exceeding the quota for Automation accounts in your subscription in the region.

Step 2 - Set Identity

For aa-spark-automation0, complete the following steps:

  1. Expand the Account Settings section and select Identity from the left navigation
  2. Under the System assigned tab, turn the Status to On
  3. Save the change
System Assigned Identity

Step 3 - Import Certificate

  1. Expand the Shared Resources section and select Certificates from the left navigation
  2. Click on + Add a certificate
Add Certificate
  1. Fill in the values shown below in the table, and click on Create
Name Value
Name appreg-spark-sendmail-cert
Description The certificate used for the rb-spark-sendmail-toDL
Upload a certificate file The .pfx certificate file Private Key Required
Password The password of the certificate
Exportable No
Upload Certificate

Step 4 - Configure aa-spark-automation0 SAMI App Role Assignment

This script configures the aa-spark-automation0 SAMI automation account App Role Assignment2 of Exchange.ManageAsApp.

  1. Refer to the workbook spreadsheet and copy the v_subscriptionname value for the $subscriptionName variable
  2. Update the subscription name in the script
  3. Review the Connect-AzAccount lines and uncomment the appropriate connection string based on your tenant
  4. Review the Connect-MgGraph lines and uncomment the appropriate connection string based on your tenant
  5. Run the script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

# ONLY RUN THIS FOR THE aa-spark-automation0 SAMI (no other automation account updates the DLs)
# This grants our system assigned managed identity Exchange.ManageAsApp role
# This is required for the SAMI to update the Exchange Distribution Lists
# Used by the rb-spark-manage-dl-notifications runbook

# Set the subscription name
# Reference v_subscriptionname value from the workbook spreadsheet
$subscriptionName = "<subscription_name>"

# Connect to Az
Import-Module Az.Accounts
Import-Module Az.Resources
#Connect-AzAccount -Subscription $subscriptionName;
#Connect-AzAccount -EnvironmentName AzureUSGovernment -Subscription $subscriptionName;

# Get the SAMI Info

$mi =  Get-AzADServicePrincipal -DisplayName "aa-spark-automation0"

# Connect to MgGraph
Import-Module Microsoft.Graph.Authentication
Import-Module Microsoft.Graph.Applications
#Connect-MgGraph -Scopes "AppRoleASsignment.ReadWrite.All","Application.Read.All"
#Connect-MgGraph -Environment UsGov -Scopes "AppRoleAssignment.ReadWrite.All","Application.Read.All"
#Connect-MgGraph -Environment UsGovDoD -Scopes "AppRoleAssignment.ReadWrite.All","Application.Read.All"

<#

InteractiveBrowserCredential authentication failed:

Toggle WAM to True, then back to False

Set-MgGraphOption -EnableLoginByWAM $true
Set-MgGraphOption -EnableLoginByWAM $false

#>

# Get the service principal for the Exchange Online application.
 
$exoSp = Get-MgServicePrincipal -Filter "AppId eq '00000002-0000-0ff1-ce00-000000000000'"

# Set the fixed App Role ID for the Exchange.ManageAsApp

$appRoleId = "dc50a0fb-09a3-484d-be87-e023b12c6440"  # This is the fixed ID for Exchange.ManageAsApp

# Assign the app role to the automation account SAMI
 
New-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $mi.id -PrincipalId $mi.id -AppRoleId $appRoleId -ResourceId $exoSp.Id

Continue to Restricting the Service Principals

References

  1. Reference Link ↩

  2. Reference Link ↩