Nexosis logo

Startup company Nexosis is looking to optimize the time a new feature takes going from ideation and into their customers’ hands, using the DevOps practices of Infrastructure as Code, continuous integration, and automated testing, among others.

In working with Nexosis, we started with value stream mapping (VSM) the current process for taking “customer stories” through planning, development, and testing, and into production. The VSM and the parties involved highlighted areas of lead time that could be improved by reducing or eliminating the number of manual steps, by re-creating and automating the development and test infrastructure, and by adding more automation to their testing processes.

The technologies discussed include Azure Resource Manager template-driven deployments, Azure Automation Desired State Configuration (DSC), virtual machine (VM) scale sets, and automated testing. The core team includes LT Lead David Washington (@dwcares), who owns the relationship with CTO and co-founder Jason Montgomery at Nexosis; TE Leads Kevin Remde (@KevinRemde) and Brian Sherwin (@bsherwin); and Dev Lead Jennelle Crothers (@jkc137)

Customer profile

Founded in early 2015, Nexosis is a startup company of thirteen employees based in Columbus, Ohio. (See their full team and investors at Our Team.) They are a graduate of the TechStars Retail accelerator program. Their main product, Axon, is an artificial intelligence platform that “ingests multiple data sources to create actionable insights,” ranging from demand forecasting to event and promotional impact analysis. Axon can take advantage of standard input feeds, such as sales transactions or traffic counts, and uses self-learning algorithms that adjust to a customer’s baselines and trends. With this information, Axon can provide forecasts to make better-informed decisions.

Diagram of value proposition for Axon

For example, Axon can combine data from normally unrelated sources to make predictions for the restaurant and hospitality industry.

Diagram of Axon data flow

Problem statement

Axon is currently hosted in Amazon Web Services (AWS). Before our work with them, the Nexosis technical leadership came to us already convinced of the benefits of moving their software as a service (SaaS) product to Azure, initially as a lift-and-shift of their infrastructure as a service (IaaS)–based workloads into Azure Virtual Machines. What they originally wanted was a way to automate the creation and re-creation of infrastructure from development through testing to production.

Nexosis provided our team with a list of items to address in our meeting:

  • Increase our integration testing coverage
  • Add automated UI testing
  • Improve hand-off between UX (Eric) and engineering
  • Minimize manual intervention
  • Set metrics for continual improvement
  • Continuous improvement in efficiency, quality, accuracy, and speed of our delivered solutions
  • Better measurements of improvements (such as delivery efficiency) over iterations

Why move to Azure from AWS?

Although they are still running much of their environment in AWS, Nexosis considers themselves primarily a “Microsoft shop,” using .NET, Visual Studio, and Windows Server for hosting much of their application. They are moving to containerization for their Linux workloads, and they’re actively investigating Azure Functions and Azure Service Fabric to support future improvements.

According to Jason Montgomery, Nexosis decided to move from AWS to Azure for several reasons. One technical example involves the fact that Lambda functions in AWS don’t support .NET code. They were able to use Python instead but would prefer to use .NET. “The ability to take a piece of .NET code and execute it as an Azure function is very appealing,” says Jason.

Also, the security of account passwords for VM authentication was rather suspect in AWS CloudFormation when compared to Azure Resource Manager templates. Passwords in CloudFormation files persist in clear, unprotected text that you can read on the host, whereas in Azure it’s easy to secure the password string so that it is not stored in clear text. “It’s not that Amazon doesn’t have some of that,” says Jason, “but it really felt like to use them you had to go back to implementing your own from scratch. So I thought some of the security models in the [Azure] cloud were better.”

But the main reason they went with Azure was because of their positive experience with Microsoft as a trusted advisor and partner, in contrast to a lack of interest or support from other providers. Jason says,

“Just from a startup-friendly responsiveness, giving us resources to help us is huge. Helping us solve problems more quickly. And it really helps propel a lot of our stuff a lot quicker because we don’t have to go start from zero and try to figure it out for ourselves.”

Solution, steps, and delivery

For two days in Columbus, Ohio, the team from Microsoft worked with Nexosis CTO Jason Montgomery and Chief Engineer Jeff Gabriel.

The first part of our first day included Kevin, Brian, and Jennelle from Microsoft; Jason and Jeff from Nexosis; and brand-new Nexosis Executive Vice-President Nat Milburn. We discussed the areas of interest provided by Nexosis and presented them an overview of the VSM process and benefits.

Photo of the meeting room

For the remainder of the day, we focused on mapping their product and feature lifecycle, working back from the customer end.

After mapping their process, process times, and lead times, we marked where areas of waste were recognized and noted where possible solutions (in the form of DevOps best practices) could be applied. For example, the creation of the infrastructure stack for dev and test was a manual process, where heroics (one guy) were doing all the work. The Infrastructure as Code DevOps practice was recommended here, and what was a five-day process (with a ten-day lead) time could be reduced from days to hours (or even minutes).

Photo of wall-sized sketch of value stream map

Completed value stream map

Conclusion

After our VSM on day one and some day-two hacking on examples, solutions, and further discussion, we were able to add areas of further investigation to their list of items they wanted to address:

  • Increase our integration testing coverage
    • Automated testing and test frameworks
  • Add automated UI testing
    • Automated testing and test frameworks
  • Improve hand-off between UX (Eric) and engineering
    • Didn’t really address UX process handoff; perhaps some spec-level tests can be developed at that early stage
  • Minimize manual intervention
    • Azure Automation best practices
    • Infrastructure as Code at scale
    • Addressing heroics
  • Set metrics for continual improvement
    • VSM process time and lead time estimates measured
    • Goal: re-visit after recommendations have been implemented
    • Quality
    • Code coverage
  • Continuous improvement in efficiency, quality, accuracy, and speed of our delivered solutions
  • Better measurements of improvements (such as delivery efficiency) over iterations

As mentioned, we took time on day two to do some hacking on proof-of-concept examples for Nexosis. To deploy and re-deploy their development, test, and eventual production infrastructure, they wanted to leverage Azure Resource Manager templates, Azure Automation DSC, and Azure script extensions. See the next section (Source code) for Kevin’s working sample, which creates a scale set of load-balanced virtual machines, creates an Azure Automation runbook and adds a DSC configuration, and applies the DSC configuration to the machines in the scale set. (Nexosis had already created a DSC configuration for their web server, so we were able to use theirs pretty much as is.)

Although our main work involved automating the infrastructure creation, we also investigated how to get Octopus installed and configured on the infrastructure servers. Nexosis deploys their application components by using Octopus Deploy, and they have an existing PowerShell script that installs and configures this. In our work with Nexosis, I was able to demonstrate the use of Chocolatey to automate the installation even further. They are investigating this further, but even without Chocolatey doing the installation, they also have the demonstrated option of launching their automation script through an Azure Virtual Machines extension.

Another important finding from the VSM and our discussions with Nexosis was the goal of streamlining the testing process with more automated functional testing in addition to their current unit testing processes. We plan to schedule another session to cover tools and techniques that would best fit the goals of the team.

Photo of results lists

Following the exercise, we received this note from Jeffrey Gabriel:

Hi Kevin,

I really appreciated the time you and the team spent with us. I particularly benefited from the VSM exercise as it helped visually just how much our most constrained and “heroic” resource was really spread out across the entire stream. This was helpful to see as it pointed to where automation of our infrastructure would bring the greatest value.

I have also been able lean on the tools you introduced to us as I have been putting together an automation script for our entire Atlassian tool suite environment. Identification of the PowerShell ISE, the home of the quickstart templates, and tips on best practices have all been helpful.

Thanks again for spending the time with us.

Jeff

Source code

To launch the sample Azure Resource Manager template deployment, Kevin created the following script (available on GitHub as Kevins Deployment Script.ps1). When edited for a few user-specific items and values, and with the PowerShell cmdlets for Azure installed, the running script logs the user in to Azure, populates variables, and launches the deployment.

This particular execution uses GitHub-based template and parameter files, but it can be edited to use any of the various forms and options for the New-AzureRmResourceGroupDeployment PowerShell cmdlet.

# EDIT THIS!
# Put your subscription name in a variable.  
# This is really only needed if your credentials are authorized to access multiple subscriptions.  
# If you only have one subscription, a simple "Login-AzureRmAccount" command will suffice.
#
$azureAccount = "KevRem Azure"
# $azureAccount = "Visual Studio Ultimate with MSDN"

# Login
Login-AzureRmAccount
Get-AzureRmSubscription -SubscriptionName $azureAccount | Select-AzureRmSubscription 

# EDIT THIS!
# Set the path to where you've cloned the NTestDSC contents.
# Important: Make sure the path ends with the "\", as in "C:\Code\MyGitHub\NTestDSC\"
# $localAssets = "C:\Code\MyGitHub\NTestDSC\"

# Datacenter Region you want to use.  
# Note that some regions don't yet support Azure Automation. You'll get an error if you pick one of those.
$loc = "East US 2"

# Collect digit(s) for generating unique names
#
$rnd = Read-Host -Prompt "Please type some number for creating unique names, and then press ENTER."

$rgName = 'RG-WebScaleSet' + $rnd
$autoAccountName = 'webAutomation' + $rnd

New-AzureRmResourcegroup -Name $rgName -Location $loc -Verbose

New-AzureRMAutomationAccount -ResourceGroupName $rgName -Name $autoAccountName -Location $loc -Plan Free -Verbose

$RegistrationInfo = Get-AzureRmAutomationRegistrationInfo -ResourceGroupName $rgName -AutomationAccountName $autoAccountName

$NewGUID = [system.guid]::newguid().guid

# This deployment requires pulling remote files, either from Azure Storage (Shared Access Signature) or from a URL like Github.
#
$assetLocation = "https://raw.githubusercontent.com/KevinRemde/NTestDSC/master/"

# Setup variables for the local template and parameter files..
#
# $templateFileLoc = $localAssets + "azuredeploy.json"
# $parameterFileLoc = $localAssets + "azuredeploy.parameters.json"

$templateFileLoc = $assetLocation + "azuredeploy.json"
$parameterFileLoc = $assetLocation + "azuredeploy.parameters.json"

$configuration = "AxonWebServer.ps1"
$configurationName = "AxonWebServer"
$nodeConfigurationName = $configurationName + ".localhost"
$configurationURI = $assetLocation + $configuration

$moduleName = "xNetworking"
$moduleURI = $assetLocation + $moduleName + ".zip"

# Get a unique DNS name
#
$machine = "kar"
$uniquename = $false
$counter = 0
while ($uniqueName -eq $false) {
    $dnsPrefix = "$machine" + "dns" + "$rnd" + "$counter" 
    if (Test-AzureRmDnsAvailability -DomainNameLabel $dnsPrefix -Location $loc) {
        $uniquename = $true
    }
    $counter ++
} 

#
# For this deployment I use the GitHub-based template file, parameter file, and additional parameters in the command.
# 
New-AzureRmResourceGroupDeployment -Name TestDeployment -ResourceGroupName $rgName `
    -TemplateParameterUri $parameterFileLoc `
    -TemplateUri $templateFileLoc `
    -domainNamePrefix $dnsPrefix `
    -registrationKey ($RegistrationInfo.PrimaryKey | ConvertTo-SecureString -AsPlainText -Force) `
    -registrationUrl $RegistrationInfo.Endpoint `
    -automationAccountName $autoAccountName `
    -jobid $NewGUID `
    -nodeConfigurationName $nodeConfigurationName `
    -moduleName $moduleName `
    -moduleURI $moduleURI `
    -configurationName $configurationName `
    -configurationURI $configurationURI `
    -Verbose 

# Later if you want, you can easily remove the resource group and all objects it contains.
#
# Remove-AzureRmResourceGroup -Name $rgName -Force

Additional resources

  • The GitHub repo containing the sample Resource Manager template deployment and scripting: KevinRemde/NTestDSC
  • The VSM diagram in Visio form: Nexosis VSM