Microsoft joined RockStep to develop a multitenant IoT solution that reduces operating and maintenance costs for their clinical trial customers. Research facilities have thousands of units to house animals, and these units need to be checked on a regular schedule to ensure the conditions are within compliance. RockStep is teaming up with the University of Michigan to run a pilot IoT program that monitors key environmental measurements in real time and will proactively alert technicians when conditions reach a threshold range. By reducing the number of visits to each unit, they can cut maintenance and operating costs and improve the overall health of animals.

RockStep is using a Raspberry Pi 3 running Windows 10 IoT Core to monitor environment values such as humidity and ammonia (NH3) and send messages to Microsoft Azure IoT Hub. Azure IoT Hub is connected to an Azure Stream Analytics job that processes the values to create real-time alerts that are sent to an Azure Service Bus where a process from RockStep’s Climb product sends alerts to technicians. The messages are also routed to Azure Event Hubs where an Azure function processes the device readings to determine the tenant that the data belongs to and store the values in the correct storage location.

Key technologies used

Core team


picture of team working


Customer profile

RockStep Solutions creates world-class scientific data management tools for research. Its innovative software system Climb is designed to transform and modernize information management in a laboratory setting. The RockStep team’s experience comes directly from years of working at the Jackson Laboratory in Bar Harbor, Maine. RockStep’s goal is to create software that enables science by leveraging the team’s experience working in research laboratories to build software that is easy to learn, usable anywhere, and truly valuable to research.


rockstep logo


Problem statement

Monitoring the environmental measurements in a research lab is costly. Currently, lab technicians need to visit the housing units every three days to change materials to ensure the measurements remain within compliance even if the readings are within normal range. Frequent changing of materials leads to higher labor and material costs and the change rate also has a direct effect on animal stress.

By creating an IoT system that monitors the measurements levels in real time and notifies technicians when a housing unit needs to be changed, RockStep’s Azure IoT solution provides cost savings as high as 33%. In a laboratory with 50,000 housing units, this can save more than $1 million. Beyond cost savings, the system can reduce the housing unit change frequency and reduce overall stress on the animals.

Another challenge RockStep faced was building a multitenanted IoT solution. RockStep designed its current Climb product as a multitenanted Azure solution, and RockStep is directly integrating Climb with the IoT monitoring solution. As we built the solution over the course of the hackfest, we solved the problem of having to segregate data into separate databases and storage locations.

“One of our biggest concerns was ensuring that data collected within one animal room but spanning multiple researchers could be routed to the correct tenant data stores. The Microsoft team understood this issue, and we worked together to find a robust solution using the latest Azure technologies. We feel that we can very easily extend the model we developed to handle more complex scenarios. Everything came together beautifully.” —Abigail Ames

Solution and steps

RockStep and Microsoft teamed up to create a “proof of concept” that would simulate the key measurements that will be collected from the IoT devices, which will be located on the research housing units. The data is then sent from the devices to an Azure backend where alerts are generated for abnormal events and aggregated data is stored for insight into the overall status of the housing units. The solution was developed over the course of a few days at a hackfest held at the Microsoft office in Burlington, Massachusetts.

“Working with Microsoft was great. I was really impressed with how much we were able to accomplish in a short time by using Azure cloud resources and Microsoft’s IoT technology offerings. At every point in the technology stack, including the IoT devices, Microsoft had a solution available that addressed development, deployment, scaling, and management.” —Keith Sheppard

The overall architecture is as follows.


architecture diagram


The simulated device sends an 800-byte packet every second to the Azure IoT Hub where an Azure Stream Analytics job processes the data. The data has a device ID value that will be used by the rest of the system to determine the tenant that the device belongs to.

The device is communicating to Azure IoT Hub using the AMQP protocol over a TLS connection. Each device has its own unique access token that identifies the individual device. This token is scoped to allow the device to send data to Azure IoT Hub and does not allow any management of the device registry. In addition, Azure IoT Hub allows for the ability to disable individual devices via the Management Portal in the event a device is compromised. You can read more about security at IoT Hub Security from the Ground up.

The Azure Stream Analytics job has three queries: a raw data query, an aggregated query, and an alert query.


azure stream analytics queries diagram


The raw telemetry data is routed to an event hub where an Azure function is processing the data and adding the tenant information into the system. Because the data needs to be separated and stored in different locations based on the tenant information, the Stream Analytics query could not store the data directly to Azure Blob storage like many example designs we found. From the Azure function, the data is routed to different folders within a Blob storage container.

The raw telemetry Stream Analytics query was very straightforward to implement:


-- Raw data
SELECT * INTO rawtelemetry FROM iothub


The aggregated data uses the Tumbling Window Stream Analytics feature to calculate the Max, Min, and Mean values for each measurement over a given timeframe grouped by device ID (for proof of concept we kept the value low). The data flows into an event hub being processed by another Azure function that uses the tenant information to determine which SQL Database table (in the existing Climb system) the data will be stored in. The aggregated data is used within the Climb system to give basic analytics on the housing units over time.

The aggregated ASA query:


-- processed data query (all measurements not shown in sample)

SELECT
    DeviceID, MAX(Humidity) AS MaxHumidity, MIN(Humidity) AS MinHumidity, AVG(Humidity) AS MeanHumidity, COUNT(*) AS Count
INTO
    processedtelemetry
FROM
    iothub
GROUP BY
    DeviceID, TumblingWindow(second, 30)


The alerts Stream Analytics query was written to test if any of the measurements were over or under a specified threshold value specified by the device ID. If over or under the threshold, the entire telemetry message would then be written to an Azure Queue that is being monitored by the RockStep Climb Alert system. The RockStep Alert system was already built as part of the Climb system.

The thresholds values are consumed by the Stream Analytics query using Azure Stream Analytics Reference data as an input into the Azure Stream Analytics job. The reference data is joined to the incoming IoT Hub streaming device data. The team decided the thresholds values would be specified by tenant allowing for the end customer to have the most flexibility when configuring the system in their research lab.

For the prototype phase, we directly uploaded the reference data to the Azure Blob storage location used by Stream Analytics. RockStep will be adding functionality to the Climb product that enables customers to link the device ID to their account and customize the thresholds. This will require a refresh to the reference data by time stamping the blob name.

The alerts Stream Analytics query:


-- Alerts query  (all measurements not shown in sample)

SELECT
    *
INTO
    alerts
FROM
    iothub JOIN thresholds ON iothub.DeviceID = thresholds.deviceID
WHERE
    iothub.Humidity > thresholds.humidityUpper OR
    iothub.Humidity < thresholds.humidityLower OR

    iothub.NH3 > thresholds.nh3Upper OR
    iothub.NH3 < thresholds.nh3Lower OR


To enable the multitenanted scenario, Event Hubs and Azure Functions were required to separate the telemetry data by tenant. A sample of the Azure function that determines the tenant based on the device ID and then outputs it to a dynamic location is as follows:


#r "Newtonsoft.Json"
#r "System.Data"
#r "System.IO"
#r "Microsoft.WindowsAzure.Storage"
using System;
using System.Data.SqlClient;
using System.IO;
using System.Configuration;
using Newtonsoft.Json;
using Microsoft.Azure; // Namespace for CloudConfigurationManager
using Microsoft.WindowsAzure.Storage; // Namespace for CloudStorageAccount
using Microsoft.WindowsAzure.Storage.Blob; // Namespace for Blob storage types

// Main entry point
public static void Run(string myEventHubMessage, TraceWriter log)
{
    if(!string.IsNullOrEmpty(myEventHubMessage)){
       ProcessMessages(myEventHubMessage, log);
    }
    else
    {
        log.Info($"No message");
    }
}

public static void ProcessMessages(string messages, TraceWriter log){
    log.Info($"Messages to process: {messages}");

    var parsedMessage = JsonConvert.DeserializeObject<ProcessedMessage>(message);
    var tenantid = GetTenantId(parsedMessage, log);

    CreateBlobDataByTenant(message, tenantid, log);
}

public static void CreateBlobDataByTenant(string message, string tenantId, TraceWriter log){
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["storageconnection"]);
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

    // create blob container by tenant Id
    CloudBlobContainer container = blobClient.GetContainerReference("tenant" + tenantID);
    container.CreateIfNotExists();

    //store records by date
    CloudAppendBlob appendBlob = container.GetAppendBlobReference(DateTime.Now.Date.ToString() + ".log");
    appendBlob.CreateOrReplace();

    appendBlob.AppendText($"{message}{Environment.NewLine}");
}

public static string GetTenantId(ProcessedMessage message, TraceWriter log){
    string tenantID="";
    var str = ConfigurationManager.ConnectionStrings["sbirpilot_db"].ConnectionString;

    using (SqlConnection conn = new SqlConnection(str))
    {
        string deviceID=parsedMessage.deviceid;
        conn.Open();
        var text = "SELECT TenantID FROM TenantDevice WHERE DeviceID=@DeviceID";

        using (SqlCommand cmd = new SqlCommand(text, conn))
        {
            cmd.Parameters.Add(new SqlParameter("@DeviceID", deviceID));
            tenantID = cmd.ExecuteScalar().ToString();
            log.Info($"Tenant ID is {tenantID}");
        }           
    }
}

public class ProcessedMessage {
    public string maxtemperature{get;set;}
    public string mintemperature{get;set;}
    public string meantempurature{get;set;}
    public string count{get;set;}
    public string deviceid{get;set;}
}


Conclusion

The “proof of concept” is now complete and RockStep is working to build the Raspberry Pi 3 hardware device that we simulated. RockStep is also working to make sure the data collected is fully integrated into the Climb system so customers can best leverage the IoT solution with their existing software. The entire solution will be tested with a pilot at the University of Michigan. Learnings from the pilot will be integrated back into the solution and, because of the multitenanted solution, will then become available for other RockStep customers.

“The Azure IoT Hub solution allows us to easily connect real-time streaming sensor data with our existing web-based animal colony management system. The result is a seamless pipeline for collecting environmental data, analyzing them, and taking direct actions to improve the overall health and safety of the animals. There is no other vendor in our space who is offering this type of seamless IoT integration. In addition, there is a clear path for us to extend this pipeline to include additional high-value Azure services such as Machine Learning analytics.” —Abigail Ames

General lessons

The major learning the team had while implementing the solution was how to incorporate the multitenanted requirement into the solution. When we first whiteboarded the solution, we assumed we could dynamically specify the table for the Azure SQL Database outputs and the path in the Blob storage during the running of the Stream Analytics query. After we had implemented the Raspberry Pi simulator and could get data into the IoT Hub, we realized we would need to come up with another solution.

After a thorough discussion, we landed on the architecture described earlier. We considered integrating the tenant information into the telemetry via an Azure Stream Analytics query using reference data, as we did with the thresholds, but decided that the solution of using an Azure function offered greater flexibility. We had to create the Azure function anyway to be able to dynamically specify tables and blob locations. Doing it at that time simplified updating the tenant information via the Climb system.


white boarding session


Opportunities going forward

RockStep plans to integrate a Power BI Embedded dashboard into the Climb system that will leverage the data that is being collected to help the customers of Climb get better insight into their research facilities. RockStep is also excited about integrating Azure Machine Learning as part of the system to create a predictive model to better improve maintenance.

“At our hackathon, we solved the problems of data acquisition, aggregation, and threshold-based alerting in a multitenanted configuration, but it feels like we’ve just scratched the surface of what can be done with the data to make it more meaningful and helpful to customers. I’m looking forward to exploring the Azure Machine Learning components (especially for Anomaly Detection) and Power BI for visualization to get the most value out of the data we’re collecting.” —Abigail Ames

Additional resources

As the team developed the solution, we came across several key resources:

To learn more about IoT, see: