The managers of Dublin’s Croke Park Stadium know that being a good neighbor means ensuring its events have minimal impact on those who live nearby. That starts with environmental sound monitoring. When it comes to decibels, Croke Park must stay within the parameters established by the Dublin City Council. Prior to this Smart Stadium initiative, an independent third party would record the noise levels and let the stadium know after the fact whether it was in compliance. Now this monitoring is done in real time.

An automated solution solves a number of problems for the Gaelic Athletic Association (GAA), which owns and manages Croke Park:

  • Reduced overhead in sound monitoring: The current solution is very manual and requires significant effort throughout a concert to record results.
  • Sound data can be disseminated through multiple channels; for example, a website, a publicly accessible app or a dashboard accessible by key personnel.
  • Removing the noise pollution from external forces that could be mixed in with the internal sounds.

The Smart Stadium project is an ongoing collaboration between the GAA, Dublin City University (DCU), and Microsoft. This case study outlines the next phase within the Smart Stadium project by leveraging Microsoft Power BI Embedded and Machine Learning for displaying sound data monitoring within a smart stadium.

Key technologies used

The following technologies were used during the implementation of this solution:

Core team

Microsoft:

  • Mike Myers, Principal Technical Evangelist, Microsoft Ireland
  • Niall Moran, Senior Technical Sales Manager, Microsoft Ireland

Dublin City University and Intel:

  • Suzanne Little, Researcher Team Lead, Dublin City University
  • Camille Ballas, Researcher and Developer, Dublin City University
  • Dian Zhang, Researcher and Developer, Dublin City University
  • David K. Prendergast, Anthropologist, Intel Ireland

Smart Stadium

After completion of the initial Smart Stadium project, an opportunity was envisioned to build a new solution that would leverage sound data, both internal and external, from the stadium into the Azure platform. (Internal refers to sound data recorded within the stadium and external refers to all types of sound data outside of the stadium, including noise pollution from traffic and general city life.) Within this new project, a Machine Learning model will be developed that will remove the noise pollution and present an extended and more accurate data visualization in the form of Power BI Embedded reports. These reports will be embedded within an Azure App Service and will access its data via a Web API and a Machine Learning API.

Another key requirement for this solution was the importance of strategically positioning the microphones within the stadium. To understand noise levels for both crowd cheer and general city noise, microphones must be located both within the stadium bowl and externally. For this reason, four microphones were deployed—two on the east side of the stadium and two on the west side. Each side had one microphone inside the stadium and one outside.

The success of this project has now given rise to this solution, along with a number of others related to the Smart Stadium, moving into the next stage of development by moving into production. It’s also worth noting that other stadiums from around the world are interested in this solution. An example is Arizona State University, which is redeveloping its Sun Devil Stadium, leveraging the learnings from this Smart Stadium project.

Customer profile

The Gaelic Athletic Association (GAA) is Ireland’s largest sporting organization with 2,500 clubs in Ireland and 500,000 members. It is celebrated as one of the great amateur sporting associations in the world. Part of the Irish consciousness, it plays an influential role in Irish society that extends far beyond the basic aim of promoting Gaelic games. Every summer the intercounty All-Ireland Championships in hurling and football capture the attention of the Irish public, and regional towns heave with the arrival of large numbers of supporters and the color, noise, and excitement they bring. Approximately 1.5 million people attend the GAA Championships from May to September at Croke Park Stadium.

Croke Park Stadium plays host to iconic moments in Irish sports and history and to major cultural and international events. It functions as home to one of the world’s leading Internet of Things (IoT) programs, with Croke Park the test bed for everything from pitch management and crowd control to micro-weather wind circulation.

Problem statement

The Smart Stadium project was about the ingesting of telemetry data from sound-monitoring sensors. While this project was very successful, it was understood that the telemetry data included both internal and external data sources. As such, within this project we set out to remove the noise pollution from the data feeds. Once the data is corrected, we needed a way to be able to present the data within the web dashboard. To achieve this, we used several technologies that are documented in this case study. The core problems that we needed to address included the following:

  • Correct the accuracy of the ingested data.
  • Present the data in an easy-to-understand format.
  • Highlight spikes in the data.
  • Monitor progress over time.

The following section outlines how we approached this problem and built out a solution to address it.

Solution, steps, and delivery

Step 1: Solution envisioning

The web application (dashboard) and its associated services all run within the Azure platform as an app service. The app service connects to its data source via a REST API, which in turn connects to a SQL database that uses several tables to hold the received data that comes from a Stream Analytics job.

The solution is initially composed of several sound-recording devices that send sound data to a field gateway that in turn pushes an aggregated message up to an Azure IoT Hub via the AMQP protocol. A Stream Analytics job then performs a number of queries on this data and pushes the data into the required table—either a rolling average table or just standard data that is envisioned using an Azure SQL database. This is mainly used for the reporting needs of the solution and is consumed using several Power BI reports that are embedded within a web app running as an Azure app service.

Solution Architecture


The Power BI Embedded service uses this database as its data source. The following steps were identified for the technical implementation:

  • Ingest sound data in its raw format.
  • Store the data in SQL database tables and include a rolling table (accumulated figures).
  • Create a Machine Learning model to strip out the external noises from the data, resulting in just the internal noise data.
  • Create a REST API to provide GET operations in the Power BI Embedded workspace collection to retrieve the data for consumption.
  • Set up the use of DirectQuery with Azure SQL Database so that the data can be displayed by using the Power BI Embedded REST API functionality.
  • Create .pbix reports using Power BI Desktop.
  • Upload .pbix reports to the server.
  • Create the web dashboard.
  • Include the reports in the web app for consumption.

Step 2: Create the IoT hub

The initial step involves creating an Azure IoT hub. We went with an S1 Standard pricing tier because the number of messages is expected to grow to more than 8,000 per day and we could have several gateways (devices) that would be sending telemetry data to the IoT hub. It was also agreed that 1 unit would suffice for this scenario.

While this scenario is solely about pushing sound telemetry data into the Azure platform, it could be argued that we could have used an event hub instead. We went with an IoT hub because we wanted to use the additional features that this brings, including Device Identity and Management and the ability to do Cloud to Device messaging because this will be used in future versions of the solution.

Azure IoT Hub

Step 3: Create the Stream Analytics job

Once data has been ingested into the IoT hub, Stream Analytics is used to analyze the data and create a stream of data for each use case. Each stream of data is then outputted to a table in the SQL database. There are two Stream Analytics jobs, one to handle weather data and another to handle sound. Each job has the IoT hub as input and defines a number of outputs. The table below outlines each Stream Analytics query, their inputs, queries and outputs:

Stream Analytics Job Details Table

Step 4: Create the SQL database

An Azure SQL database is used to store relevant data for analysis or display. The SQL database is a fully managed relational database service hosted on Azure and offers 99.99% SLA for availability, which helps this solution scale to a full production system. The database also supports 100 database transaction units but can scale to 4,000 units when required. The DTU is a blended measure of performance that can be used to get predictable performance.

The database tables are populated by the Stream Analytics jobs described above.

Azure SQL Database Tables

Step 5: Create the Machine Learning model and service

The team created an Azure Machine Learning model to help them remove the unwanted noise pollution from the code datasets. This was achieved by receiving data from several IoT devices (gateways) with some being internal to the stadium and some being external. The model got its datasets from the SQL database as two streams and then the removal of the external forces occurred. The resulting data was stored in a storage account that was then consumed by a Power BI report. This can be seen here:

Machine Learning Service

Step 6: Create the Web API RESTful service

The team decided to create an API to allow access to the stored data within the SQL database. This would be in the form of a RESTful service and was developed as an ASP.NET Web API using Visual Studio 2015. This contains several controllers that allow access to the data that will be consumed by the front-end dashboards of the solution.

ASP.NET Web API Application Architecture


Controllers

publicclassRealTimeSoundDataController : ApiController

{

    [SwaggerOperation("GetLatestSoundData", OperationId = "GetLatestSoundData")]

    publicIEnumerable<DeviceRealTimeSoundDataModel> Get( int minutes )

    {

        returnMvcApplication.SqlDBHelper.GetLatestRealtimeSoundData(minutes );

    }

}

publicclassRealTimeDateSoundDataController : ApiController

{

    [SwaggerOperation("GetSoundDataForDate", OperationId = "GetSoundDataForDate")]

    publicIEnumerable<DeviceRealTimeSoundDataModel> Get(DateTime date)

    {

        returnMvcApplication.SqlDBHelper.GetSoundDataByDate( date );

    }

}

publicclassRollingAverageSoundDataController : ApiController

{

    [SwaggerOperation("GetRollingAverageSoundData")]

    publicIEnumerable<DeviceRollingAverageSoundDataModel> Get(int minutes)

    {

        returnMvcApplication.SqlDBHelper.GetRollingAverages(minutes);

    }

}

publicclassRollingAverageDateSoundDataController : ApiController

{

    [SwaggerOperation("GetRollingAverageSoundDataByDate")]

    publicIEnumerable<DeviceRollingAverageSoundDataModel> Get( DateTime date )

    {

        returnMvcApplication.SqlDBHelper.GetRollingAveragesByDate( date );

    }

}


SQLDBHelper

publicclassSQLDBHelper

{

    public SQLDBHelper()

    {

    }

    ///<summary>

    /// Returns the latest information for each device over a certain time period in minutes

    /// This method looks at Realtime LAMax figures and gets the largest LAMax figure for the time period

    /// as well as the most recent LAMax data. Max(LAMax) should be used to monitor crowd cheer etc

    ///</summary>

    ///<param name="minutes"></param>

    ///<returns></returns>

    publicList<DeviceRealTimeSoundDataModel> GetLatestRealtimeSoundData(int minutes )

    {

        var dbContext = newAzureSQLDBDataContext();

        var latestTime = DateTime.Now.ToUniversalTime().AddMinutes(-minutes);

        var data = from d in dbContext.SoundDatas

                   where d.Time >= latestTime && d.LAMax != null && d.LAMax > 0

                   group d by d.DeviceId into g

                   selectnewDeviceRealTimeSoundDataModel

                   {

                       DeviceId = g.Key,

                       MaximumSoundData = (from x in g orderby x.LAMax descendingselect x).FirstOrDefault(),

                       LatestSoundData = (from y in g orderby y.Time descendingselect y).FirstOrDefault()

                   };

        var list = data.ToList();

        return list.OrderBy(x => x.Device.SortIndex).ToList();

    }

    publicList<DeviceRealTimeSoundDataModel> GetSoundDataByDate( DateTime date )

    {

        var dbContext = newAzureSQLDBDataContext();

        var data = from d in dbContext.SoundDatas

                   where d.Time.Date.Equals(date.Date) && d.LAMax > 0

                   group d by d.DeviceId into g

                   selectnewDeviceRealTimeSoundDataModel

                   {

                       DeviceId = g.Key,

                       MaximumSoundData = (from x in g orderby x.LAMax descendingselect x).FirstOrDefault(),

                       MinimumSoundData = (from y in g orderby y.LAMax ascendingselect y).FirstOrDefault()

                   };

        return data.ToList().OrderBy(x => x.Device.SortIndex).ToList();

    }

    ///<summary>

    /// Returns the data from a sliding window of 15 minutes for the time window specified in minutes

    ///</summary>

    ///<param name="minutes"></param>

    ///<returns></returns>

    publicList<DeviceRollingAverageSoundDataModel> GetRollingAverages(int minutes)

    {

        var dbContext = newAzureSQLDBDataContext();

        var devices = DeviceFactory.Instance.Devices;

        var latestTime = DateTime.Now.ToUniversalTime().AddMinutes(-minutes);

        var data = from d in dbContext.SoundDataRollingAverageModels

                   where d.Time >= latestTime && d.AVGLEQ != null && d.AVGLEQ > 0

                   group d by d.DeviceId into g

                   selectnewDeviceRollingAverageSoundDataModel

                   {

                       DeviceId = g.Key,

                       SoundData = g.OrderBy( l=>l.Time).ToList()

                   };

        // order data by device sort order

        return data.ToList().OrderBy( x=>x.Device.SortIndex).ToList();

    }

    ///<summary>

    /// Returns the data from a sliding window of 15 minutes for the time window specified in minutes

    ///</summary>

    ///<param name="minutes"></param>

    ///<returns></returns>

    publicList<DeviceRollingAverageSoundDataModel> GetRollingAveragesByDate( DateTime date )

    {

        var dbContext = newAzureSQLDBDataContext();

        var devices = DeviceFactory.Instance.Devices;

        var data = from d in dbContext.SoundDataRollingAverageModels

                   where d.Time.Value.Date.Equals(date.Date) && d.AVGLEQ != null && d.AVGLEQ > 0

                   group d by d.DeviceId into g

                   selectnewDeviceRollingAverageSoundDataModel

                   {

                       DeviceId = g.Key,

                       SoundData = g.OrderBy(l => l.Time).ToList()

                   };

        // order data by device sort order

        return data.ToList().OrderBy(x => x.Device.SortIndex).ToList();

    }

}


Step 7: Create the web dashboard

A simple web dashboard was created to present the data online. The dashboard was built using the MVC framework within Visual Studio 2015 and used Power BI Embedded to embed the Power BI dashboards. The Power BI dashboards were built using Power BI Desktop and then uploaded to Microsoft Azure to make them available to the web dashboard.

Two Power BI reports were created and uploaded to an Azure Power BI workspace. Both reports connect directly to the Azure SQL database described earlier and use several views to present the following information:

  • 15-minute rolling average sound data and maximum spikes for the last 20 minutes, 60 minutes, 2 hours, 1 day, and any specific date to review historical data. This data is represented in a single Power BI dataset as tabs (see diagram below).
  • Most recent wind speed data, average wind speed data for the current day and a time series graph of wind speed for the current day. This graph helps indicate the likelihood of a stadium skyline tour taking place.

The following diagram shows the dashboard interfaces:

Web Dashboard

Step 8: Create the Power BI workspace collection and provision

This step involved creating a Power BI workspace within the Azure portal. Each workspace of Power BI Embedded is the workspace for each customer (tenant), and we can add many workspaces in each workspace collection. The same access key is used in each workspace collection. In effect, the workspace collection is the security boundary for Power BI Embedded. We get our access key under the general area as can be seen here:

Power BI Access Keys


Now that the provisioning is done and we have our access key, we need to create a workspace in the workspace collection via REST APIs. As shown in the following figure, we have three workspaces in our workspace collection.

Power BI Workspace


The following HTTP POST Request (REST) was used to create a new workspace in our existing workspace collection. This is the POST Workspace API. We just set the access key, which we previously copied, as AppKey. It’s very simple authentication!

HTTP Request

POST https://api.powerbi.com/v1.0/collections/SmartCrokePark/workspaces

Authorization: AppKey dgdu2277db...

HTTP Response

HTTP/1.1 201 Created

Content-Type: application/json; odata.metadata=minimal; odata.streaming=true

Location: https://wabi-us-east2-redirect.analysis.windows.net/v1.0/collections/mypbiapp/workspaces

RequestId: 6665385-2yy3-406b-8501-4ehdfgh5f6da

{

  "@odata.context": "http://wabi-eu-north-redirect.analysis.windows.net/v1.0/collections/mypbiapp/$metadata#workspaces/$entity",

  "workspaceId": "55560a09-66566-4658-a8bb-9hfghghbb9d",

  "workspaceCollectionName": "SmartCrokePark"

}


The returned workspaceId is then used for all subsequent API calls. Our solution will retain this value.

The next step in this process is to create the data connection and the reports that will be embedded within our App Service. For this task, there’s no programming or code. We just use Power BI Desktop.

With Power BI Desktop, we connect to the data sources that we need within the SQL database and then shape the data using a model to create the report, which will be consumed by the front-end dashboard. Saving the reports as a .pbix file, we then upload them onto the Power BI service. Using the Power BI Desktop service helped us in streamlining the process of designing and creating the repositories and reports.

Now we need to import our previously saved .pbix reports into the Power BI workspace. Each report in a workspace corresponds to a single Power BI Desktop file with a dataset (including datasource settings). We import our .pbix file to the workspace using the Power BI API using our previously saved workspaceId. An example of this is seen here:

POST https://api.powerbi.com/v1.0/collections/mypbiapp/workspaces/32960a09-6366-4208-a8bb-9e0678cdbb9d/imports?datasetDisplayName=mydataset01

Authorization: AppKey dgdu2277db...

Content-Type: multipart/form-data; boundary="ABCtest"

-- ABCtest

Content-Disposition: form-data

{the content (binary) of .pbix file}

-- ABCtest --


This import task takes a while to run and, when complete, returns an import ID in the form of a GUID.

Embedding the reports into the dashboard, we need to secure the authorization header value because we do not want to be using our access key for security reasons; as such, we use an OAuth JSON Web Token, which consists of the claims and the computed digital signature.

For embedding our report, we must get the embed URL and report ID using the following REST API.

HTTP Request

GET https://api.powerbi.com/v1.0/collections/SmartCrokePark/workspaces/55560a09-66566-4658-a8bb-9hfghghbb9d/reports

Authorization: AppKey dgdu2277db...

HTTP Response

HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true
RequestId: d4099022-405b-49d3-b3b7-3c60cf675958

{
  "@odata.context": "http://wabi-us-east2-redirect.analysis.windows.net/v1.0/collections/SmartCrokePark /workspaces/55560a09-66566-4658-a8bb-9hfghghbb9d/$metadata#reports",
  "value": [
{
  "id": "2027efc6-a308-4632-a775-rrtrt65656",
  "name": "mydataset01",
  "webUrl": "https://app.powerbi.com/reports/5545efc6-a308-4642-a775-b9a9186f087c",
  "embedUrl": "https://embedded.powerbi.com/appTokenReportEmbed?reportId=4555efc6-a308-4632-a775-b9a9186f087c",
  "isFromPbix": false
}
  ]
}


We can now embed the report in our web app using the previous app token and here are the results, with each report running in an iFrame container.

Power BI Report for Sound Data

Conclusion

Engagement approach

One of the challenges in working on the Smart Stadium project was the various skillsets required, including:

  • Sound and weather monitoring specialists. This activity was primarily carried out by Croke Park staff and Sonitus, a specialist sound monitoring organization.
  • Gateway management, including deployment, networking, and development. Intel deployed and managed all gateways within the stadium, with GAA IT staff providing connectivity to the network and Internet, where required.
  • Stadium staff for access control and health and safety monitoring.
  • Cloud specialists to handle the ingestion and analysis of collected data. Cloud capabilities were provided by Microsoft.
  • Business intelligence and UX experts to develop dashboards and user interfaces for displaying the data in effective ways. All BI dashboarding was provided by Microsoft.
  • Data scientists to analyze data and develop predictive models to proactively act on intelligence extracted from historical data. All data science work was carried out by a team of research scientists at Dublin City University.

Governance model

Bringing all of these resources together and successfully managing the delivery of each use case was challenging and required a governance model managed by two core teams:

  • A central governance team responsible for agreeing on use cases and alignment between all parties. This team managed the budget for all delivery and provided the direction for prioritizing the delivery of specific use cases.
  • A core technical team responsible for designing and implementing solutions for each use case.

Going forward

As this project grows with more phases added to the overall solution, it is envisioned that the engagement team will grow and this will result in multiple teams, including operational teams. These learnings can then be applied to other Smart Stadium/City projects.

It is worth noting that this project along with several other planned projects are all part of a broader solution around moving these all into production. As such, this project is now moving into a production-ready phase.

Additional resource

Croke Park: Sound and weather data monitoring within a smart stadium