In this four-day Mobile DevOps hackfest, Microsoft teamed up with Statoil to hack on an existing Xamarin solution, PleaseInspect. In this case study, we describe the process we used to improve the DevOps practices and the result.

The Mobile DevOps practices that we implemented were:

  • Continuous integration (CI)
  • Continuous delivery (CD)
  • Automated UI testing

Key technologies used

Core team

The hackfest team included members from Microsoft, Statoil, and consultants from Bouvet:

  • Lars Kåre Skjørestad – Technical Lead, Product Owner, Statoil
  • Olav Kåre Vatne – Developer, Statoil
  • Arild Eikeland – Developer, Statoil
  • Bjarte Bore – Developer, Bouvet
  • Frode Rosland Hus – Developer, Bouvet
  • Chris Risner – Principal SDE Manager, Microsoft
  • Thiago Almeida – Senior SDE, Microsoft
  • Ronny Hansen – Senior Technical Evangelist, Microsoft
  • Frank Jusnes – Senior Consultant, Microsoft

Customer profile

Statoil is an international energy company with operations in 37 countries. Building on more than 40 years of experience from oil and gas production on the Norwegian continental shelf, they are committed to accommodating the world’s energy needs in a responsible manner, applying technology and creating innovative business solutions. They have approximately 22,000 employees worldwide, and are listed on the New York and Oslo stock exchanges.

Statoil has high ambitions for international growth and has business operations in 37 countries around the world, as shown in the following diagram.


Statoil business operations


Focus of the hackfest

Statoil is close to releasing three internal mobile apps for iOS and Android, all built using Xamarin.

They are also planning to implement many more mobile applications, so one important output of the hackfest is to build reusable templates and scripts for use on future projects.

Statoil has chosen Xamarin and parts of Microsoft Azure as key components in their strategy around a mobile platform. They are experimenting with Visual Studio Team Services and Xamarin Test Cloud for their DevOps pipeline.

A success in this area will help establish Xamarin as the preferred mobile framework, together with Team Services, HockeyApp, and Xamarin Test Cloud as the preferred DevOps pipeline.


On the left: Frode, Lars Kåre, Arild, and Olav. On the right: Chris and Bjarte.

The team


Problem statement

The three main areas that Statoil wanted to improve upon are:

  • Continuous integration (CI) – Every time code is checked in, a beta is built.
  • Automated UI testing – After the beta is built, an automated UI test is run.
  • Continuous delivery (CD) – Every time a beta version is approved by an automated UI test, the application is made available to beta testers through HockeyApp.

In addition, the manual steps for building a release and release candidate (RC) should compile one binary file that is released to RC testers. Upon approval from RC testers, the same binary is to be made available to the rest of the users.

Statoil had a working build definition for the app in Visual Studio Team Services, but it was a bit complex and only ran on Linux/macOS because it was written in Bash. Because this was to be reused across multiple apps, it was important that this part was optimal and reusable across apps and operating systems.

The Statoil app PleaseInspect has versions for both iOS and Android, but during the hackfest we focused on the iOS version.

Build flow diagram for PleaseInspect


The Statoil organization had presented to the developers a requirement to reduce risk in the projects. Based on that, the developers identified CI, automated testing, and CD as important parts to manage different risks, including manual errors.

The expected benefit was that the development team would spend less time on the release process and have greater confidence in the quality of the applications.

Metrics

The current situation before the hackfest:

  • Statoil had created a CI build definition in Team Services for one iOS app.
  • The lead time for releasing the app to beta testers was several weeks.

The lead time for releasing the apps through AirWatch to RC and release devices varies a lot; it could be anywhere from a couple of hours to 48 hours because the validation process of apps released to AirWatch is manual and done by a different department from the developers. However, they are working on a solution where they release directly to AirWatch through AirWatch APIs. This is not yet solved.

Solutions, steps, and delivery

The new solution includes an improved build definition, release manager, and automated UI test.

The revised build and release flow


Part 1: Using CI to build the beta test version, trigger automatic UI testing, and deploy the app

The DevOps practices we are enabling are:

  • Continuous integration (CI)
  • Automatic UI testing
  • Continuous deployment (CD)

Statoil implemented a build definition that is executed every time source code is checked in by a developer.

The CI build triggers the test release flow (see previous diagram), after which a successful build triggers a Release Manager definition. The Release Manager definition has two goals: run an automatic UI test, and then deploy the app.

The Release Manager definition includes three steps to execute the UI test by using the tool mobile-center:

  1. Install the mobile-center-cli so the release agent can execute commands against mobile-center; execute npm with arguments:

    install -g mobile-center-cli

  2. Sign in to mobile-center using build definition variables; execute mobile-center with arguments:

    login -u $(mobilecenteruser) -p $(mobilecenterpassword)

  3. Execute UI test in mobile-center; execute mobile-center with the following arguments:

       test run uitest 
       --app "xxxx/pleaseinspect" 
       --devices b372dc48 
       --app-path $(System.DefaultWorkingDirectory)/PleaseInspect-iOS-Release/drop/RC/PleaseInspect.iOS.ipa  
       --test-series "master" 
       --locale "en_US" 
       --build-dir $(System.DefaultWorkingDirectory)/PleaseInspect-iOS-Release/drop/RC/ 
       --uitest-tools-dir $(System.DefaultWorkingDirectory)/PleaseInspect-iOS-Release/drop/RC/
    


Release Manager definition


After a successful UI test, the beta build is distributed to all beta testers by using HockeyApp. Only beta testers are enrolled into HockeyApp. In the following diagram, we can see the information about the beta test version inside HockeyApp. In addition to distributing beta versions, Statoil uses HockeyApp to gather crash reports and feedback.


HockeyApp definition


Part 2: Building the release candidate and release version

This build step is activated manually by the developers when they want to build the release candidate and release versions. This step compiles two different versions from the same source code snapshot; the only difference in the binaries are the endpoints that the apps connect to.

Building the iOS version requires a Mac computer. In this case, Statoil utilized an on-premises macOS machine with the Visual Studio Team Services build agent installed. Another option is to use a MacinCloud build agent.

For details about how to do this, see Continuous Integration for iOS Apps with Visual Studio Team Services.

When building the release candidate (RC), it was important that the build process build both the RC version and the release version from the same source code. The reason for this is that when the RC version is approved (which can be some days after the code is compiled and deployed to the RC testers), there shouldn’t be any changes to the application when released to all users. So if the build of the release version is done in another build definition, there is a risk of source code changes between the compiles.

In the source code, all configuration properties are to be initialized by build parameters.

Source code parameters replaced during build


This is a problem because normally you would have unique variables in different build definitions, but because we needed one build definition for release and RC, we had to solve this another way.

We solved this by enabling multi-configuration in the build definition.

Enabling multi-configuration in the build definition


The variable BuildConfiguration contains the different builds to be executed. In this build definition, the variable contains RELEASE_ and RC_.

Some of the parameters in the source code need to be unique for the release and RC. We solved this by prefixing the variables with the BuildConfiguration variable name and then creating two versions of the variables and prefixing them with RELEASE_ and RC_. For variables that were common, we didn’t prefix the variables.


List of variables in the build definition


During the build, we need to rename the variables to make them match the source code. The script needs to work cross-platform, so it was written in Node.js.

Node script


The complete build definition looks like this.

Complete build definition


A successful build triggers a Release Manager definition, which is similar to the one described earlier. The Release Manager definition has two goals: run an automatic UI test and then deploy the app to the RC testers by using AirWatch mobile device management (MDM).

If the RC testers approve the RC build, Release Manager deploys the release build to all users through AirWatch. Statoil used the MDM tool from VMware to sideload the application into their mobile devices, so they don’t need to go through the app stores for their mobile devices.

Conclusion

Our impact

  • After the hackfest, Statoil is now able to release every week. Several factors come into play: the CI environment has removed uncertainties related to building the code due to improvements in the processes around handling the source code and to release planning.
  • After the hackfest, Statoil implemented the identical flow for Android, so now they have CI/CD and automated testing for both iOS and Android.

Opportunities going forward

  • Reuse the scripts for all the Statoil mobile apps. Three more Xamarin apps are in the pipeline, and Statoil is expecting to build many more apps soon, probably around 50.
  • Implement Infrastructure as Code (IaC). During the hackfest we started implementing IaC for the Azure backend to deploy development and QA environments and optional production environments. This is something Statoil is working on to add more DevOps practices.

General lessons

  • Microsoft is transitioning mobile apps from Application Insights to HockeyApp, so the SDK for mobile Application Insights is deprecated. We had some trouble with this because features available in Application Insights were currently unavailable in HockeyApp. This impacted the way we had to send telemetry from the app. Microsoft is working to improve this; see details in the links under Additional resources.
  • Building automated flows for compiling and testing the app enabled the development team to produce new versions more quickly with higher quality.
  • Build and release flows created during the hackfest can be reused for new applications and other mobile operating systems with minor adjustments.
  • During the process of creating IaC, we found out that the Visual Studio Team Services build agent had “insufficient privileges” when we tried to dynamically get a service principal name (SPN) attached to an Azure Active Directory application. After some digging, we found the solution:
    • If you already created the endpoint in Team Services, you can update the role of the Service Principal to Owner or some other role with the right permission. The default role, Contributor, does not have enough permissions.
    • If you haven’t created the endpoint yet, you can create the Service Principal first before creating the endpoint in Team Services. You need to change from Contributor to Owner or another higher level role; for more information, see Creating an Azure Resource Manager Service Endpoint in new Portal.

Additional resources

DevOps

HockeyApp

Transitioning Mobile Apps from Application Insights to HockeyApp

Mobile Center