Section 14 - Deployment Validation

The order of operations within the validation phase is critical. Executing validation steps out of order may result in unintended activity. This may include notifications being sent to site owners/admins too early or managing the crawl state of sites unintentionally.

In this process, the person(s) performing validation requires:

• Permission to manage/update all SPARK runbooks within all automation accounts
• Manage SQL
• Manage Function Apps
• Manage SPO and SPARK Sites
• Access to the SPARK Management Portal
• Access to the SPARK Attestation Portal
• Access to the SPARK Shared Mailbox (granted Full Control within Exchange and auto mapped within Outlook)

Remove PnP App Registration

If you setup the pnp application registration for the deployment of SPARK. Ensure you remove the application registration from Entra.

Validation 1: Email Notifications

Runbook: rb-spark-sendemail-toDL

1. Manually add the current testing admin within the DL-SPARK-SiteOwners-1 distribution list. The testing admin is the user account of the person performing the validation testing, which must be a member of the distribution list. Not all SPARK admins will be required to be a member of the distribution list, only the individual tester.
2. Execute the rb-spark-sendemail-toDL runbook within the aa-spark-automation0 account a. Monitor the Output/All Logs/Exceptions for the runbook b. Monitor the shared mailbox outbox c. Monitor the testing admin’s local inbox

Expectation: The runbook will send a message, via the SPARK shared mailbox, to the members of DL-Spark-SiteOwners-1.

If This Fails Validate The Following:

1. v_exo_organization variable
2. $certName within the runbook is correct
3. v_exo_sendEmailApp variable
4. v_exosendEmailNPE variable
5. v_MgGraph_Environment variable
6. v_exo_email_suffix variable
7. v_siteadminurl variable
8. v_helpurl variable
9. Ensure the correct certificate with Private Key has been added to the automation account Certificates.
10. Ensure the certificate shared resource name is correct and matches the $certName value of the runbook.

Validation 2: Site Collector

Runbook: rb-spark-sitecollector

1. Access the Azure Automation Accounts
2. Select the first collector aa-spark-automation0
3. Review the Variables under Shared Resources
4. Set v_siteCollectorRunning to 0
5. Set v_permissionRunbooksRunning to 0
6. Execute the rb-spark-sitecollector runbook within aa-spark-automation0
7. Monitor the Output/All Logs/Exceptions for the runbook
8. Validate the SPARK Management card for Initial Collection increases in count

This will cause the rb-spark-siteinventory-mgmt runbook to execute. This is expected behavior, which will require you to monitor the output and completion.

Validation 3: Site Inventory Management

Runbook: rb-spark-siteinventory-mgmt

1. Execute the rb-spark-siteinventory-mgmt runbook within aa-spark-automation0.
2. Monitor the Output/All Logs/Exceptions for the runbook.
3. Validate the SPARK Management card for Analyzed Sites increases in count.

This runbook will cycle itself automatically until all sites have been analyzed. We do not want to analyze all sites at this time. Once the runbook has run once and completed, manually stop the runbook once it starts the second time.

Validation 4: Distribution List Management

Runbook: rb-spark-manage-dl-notifications

1. Execute the rb-spark-manage-dl-notifications runbook within aa-spark-automation0.
2. Monitor the Output/All Logs/Exceptions for the runbook.
3. Validate the distribution lists are being populated with the site admins/owners.

If This Fails Validate And You See The Following:

• After attempting to connect to exchange, the runbook will fail with no clear explanation
• It will give you an Exception Calling Error: “login failed for user … SQL Exception”

Validate the UAMI identity setting in Azure SQL.

Validation 5: Attestation Management Page

1. Access the SharePoint page containing the Attestation Management webpart
2. Validate the cards are displayed and showing numbers

Function App API Errors

If the application is not loading, use the dev tools to find out if the api call is responding with an error. Refer to the table below for solutions based on the error.

Token Validation Failing

Using dev tools, inspect the API call for authentication. If the error message states that the API has not been consented for, then you will need to access the SharePoint Admin Center and click on API under the Advanced section in the left navigation. Validate that the appreg-spark-spoactions api permission is approved and not pending. If the api permission has been approved, validate that the appreg-spark-spoactions matches the application registration name. If not, then a custom SPFx solution must be generated for your environment.

Function App Logs

View the fx-spark-getsitecounts function app logs to see if there is an issue with SQL connection issue. Validate the SQL connection string:

Commercial/GCC

1
Server=tcp:sqldb-spark.database.windows.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;

GCC-H

1
Server=tcp:sqldb-spark.database.usgovcloudapi.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;

DoD

1
Server=tcp:sqldb-spark.database.usgovcloudapi.net,1433;Initial Catalog=sqldb-spark;Encrypt=True;MultipleActiveResultSets=True;

Troubleshooting SQL: Out of Memory Exception

• Consider running the runbook in a Hybrid Worker with 2GB of RAM.
• For the initial collection, consider scaling the database resource to Premium and revert afterwards

References

Attestation States

AO States

Continue to Scheduling the Runbooks