Power BI Embedded, IoT, and Machine Learning for brain thermal pattern recognition with BTT
This report has been updated to describe the use of Power BI Embedded in the solution. It was originally published on Nov. 17, 2016.
Microsoft worked with BTT Corp to develop an IoT solution to help with brain thermal tunnel pattern recognition, a technique based on research by Dr. Marc Abreu of Yale University. This report describes their solution.
The core project team included:
- Robert Ortega – CTO, BTT
- David Crook – Technical Evangelist, Microsoft
- Yung Chou – Senior Technical Evangelist, Microsoft
- Paul DeCarlo – Senior Technical Evangelist, Microsoft
Brain Tunnelgenix Technologies Corp (BTT Corp) is a med-tech company with offices in the United States and Brazil. It has a biological discovery patent on the Brain Thermal Tunnel and is developing several product lines around this particular discovery. The goal of all product lines is to democratize brain thermodynamics insights to improve your health.
Through continuous monitoring of brain temperature, BTT treats brain temperature like a multichannel signal and uses this to perform analysis and pattern recognition to further insights into customers’ health on a personalized basis.
One of BTT’s large marketing points is that the device is noninvasive. See the reference image of the device application, below.
BTT Corp was looking to address two key problems. First, it needed a way to take the patterns its researchers could recognize as humans and create computer algorithms that could recognize those patterns with a high degree of accuracy. Second, it needed a way to integrate these algorithms into a production application that could perform pattern recognition in real time as well as in a batch-like process when the data is sourced from a proprietary device.
The data takes multiple processes and pathways in this particular IoT solution:
- IoT Hub -> Stream Analytics -> Azure SQL Database
- IoT Hub -> Stream Analytics -> Azure SQL Database -> Power BI
- IoT Hub -> Stream Analytics -> Azure SQL Database -> Power BI Embedded
- IoT Hub -> Stream Analytics -> Azure SQL Database -> On-premises high-performance computing (HPC) -> Azure Machine Learning
Power BI played an important role in visualizing brain thermal signals captured by the BTT device. This data was exposed using internal Power BI reports as well as externally to end users using Power BI Embedded with row-level security to provide secure data access.
The ability to use advanced R-analytics in Power BI along with visualization packages added significant value to the BTT solution. Use of R allowed us to make use of advanced analytical tools including Fourier transform, frequency domain analysis, and peak/valley identification.
Traditionally, it can be difficult to securely deliver deep insights and visualizations to customer-facing applications. For example, consider the work involved in setting up an R environment and securely exposing it to users over the Internet. Power BI Embedded allowed us to quickly create secure interactive dashboards and reports for use by physicians and patients. These leverage R visualizations that are exposed through an ASP.NET web app hosted on Microsoft Azure.
- Create Power BI workspace in Azure.
- Create a report in Power BI Desktop using Azure SQL Database as data source.
- Enforce row-level security in the report where applicable.
- Upload resulting .pbix into the Power BI workspace.
- Add data source credentials to dataset.
- Obtain appropriate OAuth token.
- Embed report into web page using OAuth token.
Data is aggregated using a series of GROUP BY statements. We take data collected during a user’s sleep session to create groups by person and session ID, then sort in ascending order in accordance to time. We then employ signal-processing techniques based on frequency domain composition to achieve further analysis. BTT is currently working to evaluate the most meaningful ways to perform deeper analysis and breakdown of its data.
Data archiving is not currently employed. This feature is not necessary at this time because BTT is far from approaching Azure SQL Database limits. However, when these limits become close to being reached, BTT will need to evaluate an archiving strategy. This will likely consist of a SQL Server pool in conjunction with an Azure Data Factory to move data into Azure Blob storage for long-term storage.
Currently, no Extract, Transform, and Load (ETL) processes are in place because we are accessing data from the Azure SQL database directly using DirectQuery. At this time, very few end users are accessing and viewing Power BI Embedded reports in BTT’s system because it is still under development. However, as BTT brings more customers onboard and the load begins to exceed limits on the SQL database, it will begin moving to a SQL Server pool. At this point, BTT will query against replicated read-only non-transactional databases or archival stores. Under normal circumstances, the developers wouldn’t use the transactional store, but, given the startup scenario of BTT, the partner chose to make this architectural decision for the time being.
Below is a screenshot of a dashboard based on historical data surfaced in Power BI with some light analytics applied:
The Power BI dashboard surfaced through Power BI Embedded in ASP.NET application:
BTT CTO Robert Ortega walks through configuration of row-level security using Power BI Embedded in the ASP.NET web app:
IoT and Machine Learning
Much of the power of BTT comes from its ability to perform pattern recognition and thus a major portion of its product relies on these capabilities. Its current workforce is already trained using C++, Python, and R on their own workstations. Due to this, we wanted to enable this team to continue their research and development while simultaneously opening a path to production deployment. Alongside the team, we built a model using Python’s sklearn packages and operationalized via Azure Machine Learning. Below is a screenshot of in-progress development of a binary classification model using sklearn on a Linux workstation.
As you can see, we are using Linux, Python, and Anaconda to perform the analysis against data pulled in from the Azure SQL data source. This was purely a preference choice, but the process is loosely coupled enough to allow you to use nearly anything. The full process for operationalizing the model into Azure Machine Learning is pictured below.
One of the more interesting parts of this experiment is that we were able to separate model training and inference away from feature extraction. This enabled the ability to stack-rank built-in Azure Machine Learning models against each other as well as custom models using the same features we extracted on the workstations during experimentation. This process enables two things. First, you can iterate just Azure Machine Learning models for a most-likely best model to pursue while also custom developing other models should the exercise be deemed necessary and still stack it against many of the common algorithms.
We then convert the experiment into a production model, which was a simple matter of choosing the endpoints and doing some basic cleanup. The following is an image of the production experiment.
Finally, after this process, we have an endpoint that can be integrated and tested in a variety of ways. One of the easiest ways to test is using the Excel endpoint provided. Below, you can see our testing of a segment of data against the endpoint to ensure we received a negative classification.
The BTT solution is an IoT solution leveraging the power of the Microsoft IoT Suite. It is an implementation following the Microsoft Azure IoT Reference Architecture Guide. A higher-level explanation of this type of solution can be found here. The key difference between this solution and many others is the absence of big data infrastructure and the fact that data analysis is done both on-premises and in the cloud. SQL is sufficient to handle current loads. BTT is evaluating SQL Data Warehouse as a big data platform when workloads increase beyond the capabilities of the current Azure SQL data source.
Device used and code artifacts
- The device used is proprietary, but based on a Texas Instruments embedded microcontroller platform with built-in Bluetooth. The device synchronizes with a client Windows device. The Windows device is built using the Universal Windows Platform and C#. The C# IoT Hub SDK is used in this scenario.
- The Annotation App for Labeling Time Series Patterns can be located here.
- Step-by-step instructions for how to deploy custom sklearn models to Azure Machine Learning can be located here.
- Instructions for building a deep neural network with Theano and Lasagne and operationalizing via Azure Machine Learning can be found here.
Opportunities going forward
BTT is planning to use Microsoft technologies (Power BI) to provide BTT™ Signal reporting and analytics, including BTT™ Signal analysis for sleep doctors and specialists. BTT plans to use Azure Notification Hubs for cross-platform alerting. BTT will also start porting the BTT™ Monitor to Xamarin.Forms to deliver a cross-platform experience. BTT is also evaluating Microsoft HealthVault and Azure SQL Data Warehouse.
“The IoT Hub in conjunction with Azure Stream Analytics has made it super easy for us to ingest data from our sensors. We also use Azure ML for developing an operationalizing predictive model. The best part is that all of these environments are ready to go and we do not have to worry about the underlying infrastructure. One of my favorite features of Azure ML is the automatically generated sample code and Excel app for testing or invoking the predictive model web service.
“Productivity and, yes, power are the words that come to mind when I think about Power BI. The combination of the Power BI Desktop and Power BI Embedded with support for R is powerful and a serious productivity boost.”
— Robert Ortega, BTT CTO
The partnership between BTT Corp and Microsoft is going to help democratize the insights into your brain and thereby improve the health of the world. The ability for a small team of primarily research scientists to take advantage of these tools and build a solution of this caliber truly drives home the democratizing power of the Microsoft IoT Suite and Cognitive Services platform.