Implement Security and Compliance in an Azure DevOps pipeline

In this lab we will Create a new Azure DevOps project and populate the project repository with our application code, then we will crate a new build pipeline, install WhiteSource Bolt from the Azure DevOps Marketplace to make it available as a task and activate it. Finally then we will add WhiteSource Bolt as one of our build tasks and scan our project code for security vulnerabilities and licensing compliance issues and view the resultant report.

DevOps Course Source

This lab is used in the following courses.

  • AZ-400T05: Implementing Application Infrastructure - Module 5 Compliance and Security. The lab compliments the AZ-400 series of courses to help you prepare for the AZ-400 Microsoft Azure DevOps Solutions certification exam.

Pre-requisites:

  • An Azure subscriptions
  • An Azure DevOps account

Lab Tasks:

  • Create an Azure DevOps account and populate the project repository with our application code
  • Create a Build pipeline
  • Install WhiteSource Bolt from the Azure DevOps marketplace and activate it
  • Add WhiteSource Bolt as a build task in our build pipeline
  • Run our build pipeline and view WhiteSource security and compliance report

Estimated Lab Time:

  • approx. 80 minutes

Task 1: Create and Configure Azure Devops environment

Our first task is to create an Azure DevOps project. To do this you will need an Azure DevOps account. If you have an existing Azure DevOps account you can use that and just create a new project, as outlined from step 3 below.

  1. If you do not have an Azure DevOps (formerly known as Visual Studio Team Services (VSTS)) account, you can sign up for a free account by going to http://visualstudio.com. Then, under Azure DevOps (formerly VSTS) section click the Get Started for free link.

    Screenshot of the wbe page https://visualstudio.com with the Azure DevOps (formerly VSTS) section and Get started for free sections highlighted.

  2. Sign in with your Microsoft account. If you do not have a Microsoft account, you can create one by clicking on No account? Create one! and following the steps complete the process to create your free Azure DevOps account.

  3. Go to http://visualstudio.com and in you Azure DevOps account click on the + Create project button.

    Screenshot of the Azure Devops account page with the + Create project button highlighted.

  4. In the create new project dialogue and click Create

    Project name: securityandcomplianceproj or any name you wish Visibility: Private

    Screenshot of the Create new project dialogue with property values highlighted.

  5. In your new project, go to Repos and under the section or import a repository click *Import

    Screenshot of the Repo under the import a repository section with Import button highlighted

  6. In the Import a Git repository dialogue select the below and click Import. We are using an existing repository that is available on GitHub at https://www.github.com/microsoft/partsunlimitedmrp

    • Source type: Git
    • Clone URL: https://github.com/Microsoft/PartsUnlimitedMRP.git

    Screenshot of the Repo under the import a repository section with Import button highlighted

  7. Once imported you should now see the files populated in your Azure DevOps repo.

    Screenshot of the Repo and then Files populated with the files

Task 2: Create a Build pipeline

  1. Go to Pipelines > Build and select New pipeline

    Screenshot of the Pipelines build with the New pipeline button highlighted

  2. In the Select a source pane use the below values and click Continue

    • Select a source: Azure Repos Git < this should be the default value >
    • Repository: < whatever you called your azure devops project i.e. securityandcomplianceproj >
    • Default branch for manual and scheduled builds: master

    Screenshot of the select a source pane with the field values highlighted

  3. In the Select a template pane scroll down to the end of the list, choose Empty pipeline, and click Apply.

    Screenshot of the select a template pane with empty pipeline highlighted

  4. Under the Tasks tab and then Pipeline in the Agent pool drop down box select Hosted VS2017

    Screenshot of the select a template pane with empty pipeline highlighted

  5. Click on the Agent job 1 section, click the + button to add a task, click on Build and scroll down to find the Gradle task. Then, click the Add button three times next to the Gradle task to add three Gradle tasks to the build pipeline. Gradle will be used to build the Integration Service, Order Service, and Clients components of the MRP app.

    Screenshot of the Add tasks build task options with Gradle highlighted

  6. Select the first Gradle task and set the below values, leaving any items not mentioned as their default values.

    • Display name: IntegrationService
    • gradle wrapper: src/Backend/IntegrationService/gradlew (either type or browse using the … button)
    • Working Directory: src/Backend/IntegrationService (either type or browse using the … button)
    • JUnit Test Results: uncheck the checkbox to Publish to Azure Pipelines/TFS, since we will not be running automated tests in the Integration Service.

    Screenshot of the first gradle task with the field values mentioned configured and highlighted

  7. Select the first Gradle task and set the below values, leaving any items not mentioned as their default values.

    • Display name: OrderService
    • gradle wrapper: src/Backend/OrderService/gradlew (either type or browse using the … button)
    • Working Directory: src/Backend/OrderService(either type or browse using the … button)
    • JUnit Test Results: check the checkbox to Publish to Azure Pipelines/TFS , and set the Test Results Files field to **/TEST-*.xml. Since the Order Service does have unit tests in the project, we can automate running the tests as part of the build by adding in a test in the Gradle tasks field.

    Screenshot of the second gradle task with the field values mentioned configured and highlighted

  8. Select the first Gradle task and set the below values, leaving any items not mentioned as their default values.

    • Display name: Clients
    • gradle wrapper: src/Clients/gradlew (either type or browse using the … button)
    • Working Directory: src/Clients(either type or browse using the … button)
    • JUnit Test Results: uncheck the checkbox in JUnit Test Results to Publish to TFS/Team Services since we will not be running automated tests in Clients.

    Screenshot of the third gradle task with the field values mentioned configured and highlighted

  9. Click on the Agent job 1 section, click the + button to add a task, click on Utility and scroll down to find the Copy Files task. Then, click the Add button twice to add two Copy Files tasks.

    Screenshot of the Add tasks build task options with Copy Files highlighted

  10. Still in the Add tasks pane under Utility and scroll down to find the Publish Build Artifacts task. Then, click the Add button twice to add two Publish Build Artifact tasks.

    Screenshot of the Add tasks build task options with Copy Files highlighted

  11. Select the first Copy Files to: task, and fill in the listed fields with the below values, leaving any items not mentioned with their default values.

    • Source Folder: $(Build.SourcesDirectory)\src (either type or browse using the … button to go to src)
    • Contents: */build/libs/!(buildSrc).?ar
    • Target Folder: $(build.artifactstagingdirectory)\drop

    Screenshot of the first copy files task with the fields mentioned highlighted

    • Note: We will copy the files we want from the build and repo and place them in a staging area on the agent, to be later picked up from there and published as build pipeline artifacts which we can later use in our release pipeline. To view more details about the variables that we are using in the tasks you can take a lok at the page Predefined build variables

  12. Select the second Copy Files to: task, and fill in the listed fields with the below values, leaving any items not mentioned with their default values.

    • Source Folder: $(Build.SourcesDirectory)
    • Contents: **/deploy/SSH-MRP-Artifacts.ps1 **/deploy/deploy_mrp_app.sh **/deploy/MongoRecords.js
    • Target Folder: $(build.artifactstagingdirectory)

    Screenshot of the second copy files task with the fields mentioned highlighted

    -Note: Try to get the formatting as it appears in the screenshot to ensure the task works smoothly. Ultimately when you check the build artifacts at the end of the lab, you will know if the task is working correctly or not and can troubleshoot or amend as you need.

  13. Select the first Publish Artifact task, and fill in the listed fields with the below values, leaving any items not mentioned with their default values.

    • Path to publish: $(build.artifactstagingdirectory)\drop
    • Artifact Name: drop
    • Artifact publish location: Azure Pipelines/TFS

    Screenshot of the first Publish Artifacttask with the fields mentioned highlighted

    • Note: In these publish tasks, we are taking the items from the agent staging area and publishing them as build pipeline artifacts for later use in our release pipeline. We will not create and deploy using our release pipeline in this lab but if you wish you may retain this build pipeline for your own testing and use later.

  14. Select the second Publish Artifact task, and fill in the input values with the following:

    • Path to publish: $(build.artifactstagingdirectory)\deploy
    • Artifact Name: deploy
    • Artifact publish location: Azure Pipelines/TFS

    Screenshot of the second Publish Artifact task with the fields mentioned highlighted

  15. Click Save and Queue, and then select Save and Queue from the drop down options to save the build pipeline and trigger a manual build, and in the Save build pipeline and queue dialogue click Save & queue again.

    Screenshot of the save and queue button with the save and queue option highlighted

  16. In the resultant Green Banner that appears click on the build number link to view the build output and logs as it progresses

    Screenshot of the green build banner with the build number link highlighted

  17. View the output as it progresses and ensure it runs successfully before continuing with the next task.

    Screenshot of the first copy files task with the fields mentioned highlighted

Task 3: Install WhiteSource Bolt from the Azure DevOps marketplace and activate it

  1. Return to our newly created build pipeline, and under Tasks go to the Agent job 1 section, click the + sign to add a task, then click on Marketplace, type whiteSource in the search dialogue, locate WhiteSource Bolt and click Gt it Free

    Screenshot of the Add tasks pane in Marketplace with WhiteSource Bolt highlighted

    -Note: WhiteSource is for build servers and WhiteSource Bolt is specifically for *Azure DevOps8, hence we will use that.

  2. You are taken to the WhiteSource Bolt page in the Azure DevOps Marketplace, click Get it free

    Screenshot of the Azure DevOps Marketplace with WhiteSource Bolt page listed with the Get it free button highlighted

  3. On the Select an Azure DevOps organization page select your Azure DevOps organization and choose Install

    Screenshot of the Azure DevOps Marketplace select an organization page with the Install button highlighted

  4. Once installed click Proceed to organization

    Screenshot of the installation complete page with proceed to organization button highlighted

  5. In your Azure DevOps project go to Pipelines and then select WhiteSource Bolt

    Screenshot of the installation complete page with proceed to organization button highlighted

  6. If prompted enter your email address and organization to activate your WhiteSource Bolt extension. You may not need to enter your email address and organization depending on your environment, you may be brought straight to the page You are using a FREE version of WhiteSource Bolt, indicating it is ready to integarte iunto your build pipelines.

    Screenshot of the installation complete page with proceed to organization button highlighted

Task 4: Add WhiteSource Bolt as a build task in our build pipeline

  1. Return to the Build pipeline we created and under Tasks go to the Agent job 1 section, click the + sign to add a task, then click on Utility, and scroll down to the end of the listed items to locate WhiteSource Bolt, then click Add

    Screenshot of the Add tasks pane under utility with WhiteSource Bolt highlighted

  2. Move the WhiteSource build task, by clicking on it and dragging it upwards, to occur after the gradle Clients task, or the last gradle build task

    Screenshot of the WhiteSource Bolt task highlighted listed after the gradle Clients build task

  3. Accept the default values in the WhiteSource Bolt task and Save the build pipeline

Task 5: Run our build pipeline and view WhiteSource Bolt security and compliance report

  1. Start a build by clicking > Queue and then click Queue again when prompted

    Screenshot of the Build pipeline with the > queue button highlighted

  2. In the resultant Green Banner that appears click on the build number link to view the build output and logs as it progresses

    Screenshot of the Build pipeline with the > queue button highlighted

  3. View the build as it progresses and ensure it runs successfully and the WhiteSource Bolt section runs successfully

    Screenshot of the build pipeline output logs

  4. When the build is completed go to Pipelines > WhiteSource Bolt and view the generated report

  5. Scroll through the report and view various elements such as

    • Vulnerability Score
    • Vulnerable Libraries
    • Severity Distribution
    • Aging Vulnerable Libraries
    • License Risks and Compliance and the associated Risk level: Apache 2.0 is listed with Risk level unknown, and two occurrences of it
    • Outdated libraries section > there is one item list gradle-REL_1.0-milestone-1, view the version details and what the recommendations are i.e. Consider updating to latest version

    • Inventory

    • Note: There is also an option to export the report.

Congratulations! You have created an open source based build pipeline and added WhiteSource Bolt to your build pipeline to check for security and compliance vulnerabilities

Summary

In this lab you have completed the following tasks:

  • Created an Azure DevOps account and populate the project repository with our application code
  • Create a Build pipeline based around open source products
  • Installed WhiteSource Bolt from the Azure DevOps Marketplace and activated it
  • Added WhiteSource Bolt as a build task in our build pipeline
  • Ran our build pipeline and viewed WhiteSource security and compliance report that resulted from scanning our application code.