Chip-ing is building a smart, connected golf ball. This ball can be located if lost on the course and can provide insight into the player’s performance by analyzing and visualizing sensor data directly captured inside the ball.

The solution consists of a mobile iOS and Android app acquiring data from the Bluetooth Low Energy (BLE) ball during play. It combines this with data from the device sensors and sends it to Azure IoT Hub. Stream Analytics then splits the data and provides the first insight in real time and moves the data to a persistent storage (SQL, DocumentDB or Table service).

Power BI provides a quick visualization of the data, giving Chip-ing key insights into how players are using their product. Chip-ing plans to implement Machine Learning with the goal of improving the locating algorithm by correlating data such as signal strength, device heading, and GPS position in order to make the ball retrieval process as efficient as possible. As a second step, the analysis of the data should also give more insights into the player’s golfing technique, possibly improving the player’s game.

Watch a demo:

Key technologies used

  • Azure IoT Hub
  • Azure Stream Analytics
  • Azure SQL Database
  • Azure Machine Learning
  • Power BI
  • iOS / Swift mobile app
  • Android / Java mobile app

Core team

Chip-ing:

  • Alex Raimondi, CTO
  • Thomas Sandel, CEO

Microsoft Switzerland:

  • Ronnie Saurenmann, Technical Evangelist
  • Sascha Corti, Technical Evangelist
  • Manuel Michaud, Evangelist Lead

Customer profile

Chip-ing AG is a startup based in Switzerland that is producing a smart, connected golf ball designed to keep from being lost and to help golfers improve their game. The ball can easily be located on the course through a mobile app, saving precious time and frustration for both the player and other players on the course. Additionally, the BLE-enabled ball contains sensors such as accelerometers that help analyze how the player is doing, with endless possibilities such as game coaching, automatic scoring, and so on.

Problem statement

Chip-ing must be able to acquire data from every ball it sells, generating massive amounts of data it needs to capture and analyze. With a potential of tens of thousands of balls being played and sending data every day, a highly scalable IoT solution is required. The ability to collect and store data is the cornerstone of this solution. It therefore needs to adapt to the potential market success of the product. Outages would be highly negative for the reputation of the company. Being a startup, an easy-to-manage and elastic cloud solution is a paramount requirement.

For the product to evolve, new ways of analyzing player data will become very important and machine learning with its potential to better predict ball location, play style, scores, and so on will become vital for future success.

Solution and steps

In this hackfest we tried to come up with an architecture and first implementation from data ingestion to data analysis in the cloud. Our goal was to get the data from the ball through the mobile app into the Azure storage as easily and efficiently as possible.

Ball

Architecture diagram

Architecture

Technical delivery

Device used

The golf ball is a Bluetooth Low Energy IoT device that includes the following sensors:

  • Accelerometer
  • Gyroscope

The ball needs to be able to withstand the forces of up to 300 G. The ball also needs to be indistinguishable to the player, compared to a standard, tournament-grade golf ball.

Dimples

Because BLE devices can’t directly connect to the Internet, a device gateway is being used in the form of the smartphone and app carried by the golf player. This provides us with the opportunity to enrich the data received from the ball’s sensors with data available from the smartphone’s additional sensors such as GPS, magnetometer, and so on.

PCB

Device messages sent

For the data ingestion, we decided to use Azure IoT Hub, a perfect fit for our needs. The pay per usage combined with great scalability and easy configuration were key factors for us. A PaaS IoT solution such as Azure IoT Hub allows us to save precious resources. No more time wasted in setting up web endpoints, queues, and so on. Microsoft takes care of all the patching and fine-tuning of the various components, with no upfront hardware investments required.

IoT Hub allows us to easily decouple the message ingestion process from the storage and analysis one. This can cope with spikes in usage without compromising the overall workflow. This simplifies maintenance but also makes it possible to evolve the different components of the solution in an independent manner. Having golfers playing worldwide makes this an all-day, every-day business. Thanks to the IoT message retention functionality, we can make changes without interrupting the production environment or losing messages.

Data is being generated at vastly different frequencies and times. For example: The RSSI (received signal strength indicator) plus the heading of the phone are generated with high frequency but only when the player is actively searching for the ball. In contrast, the accelerometer and gyroscope data from the ball are received only when the player is within BLE range of the ball.

Depending on the current activity of the player (searching for the ball, playing, and so on), different types of messages are being generated and sent to Azure IoT Hub. For optimization purposes, we don’t send RSSI data every few milliseconds but instead we package the data and send it only every few seconds, depending on the current connectivity of the device and data plan of the user.

To optimize the message, we created a hierarchical JSON format to avoid repeatedly sending static data like User ID, Ball ID, and so on. If the player is not actively searching for the ball, the app can be turned off to conserve power.

The ball has a buffer and some logic to store the data only while in use. Basically, at the end of every shoot, the data (for the current shoot) is retrieved from the ball and sent to the IoT hub. In contrast, while the player is searching for the ball, the phone data (RSSI, magnetometer, GPS) is sent with an interval of seconds to minutes.

As you can imagine, many balls being played worldwide during an average 2 to 3 hours of golf will generate quite a bit of data.

The size of the packets is not yet final—there are a lot of factors to consider such as power consumption, Internet connectivity, data plan, and more. We are trying to find the right balance for this use case.

Here is an example of the JSON payload sent from the mobile app to the IoT hub.

Receiving events… 08.12.2016 21:25:55> Device: [Mobile1], Data:[{“msgType”:1,”userId”:”7FC35737-7E17-44B2-B568-34243E5CFFE2”,”rssiData”:[{“cal”:-50,”rssi”:-53,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022635,”pwr”:8,”lat”:42.34575912127436,”lng”:7.853434837126503},{“cal”:-50,”rssi”:-52,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022635,”pwr”:8,”lat”:42.34575912127436,”lng”:7.853434837126503},{“cal”:-50,”rssi”:-55,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022639,”pwr”:8,”lat”:42.54275912127436,”lng”:7.853434837126503},{“cal”:-50,”rssi”:-56,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022639,”pwr”:8,”lat”:42.54575912127436,”lng”:7.853434837126503},{“cal”:-50,”rssi”:-56,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022640,”pwr”:8,”lat”:42.38321050887598,”lng”:7.853452977135689},{“cal”:-50,”rssi”:-58,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022642,”pwr”:8,”lat”:42.38321050887598,”lng”:7.853452977135689},{“cal”:-50,”rssi”:-66,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022643,”pwr”:8,”lat”:42.38321050887598,”lng”:7.853452977135689},{“cal”:-50,”rssi”:-65,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022643,”pwr”:8,”lat”:42.38321050887598,”lng”:7.853452977135689},{“cal”:-50,”rssi”:-62,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022646,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-61,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022646,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-64,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022642,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-64,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022642,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-67,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022650,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-66,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022650,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674},{“cal”:-50,”rssi”:-69,”id”:”08878B57-ECD3-E566-AE69-000026906CB1”,”ts”:1481022652,”pwr”:8,”lat”:42.54576299134133,”lng”:7.853437449336674}]}] 08.12.2016 21:26:54> Device: [Mobile1], Data:[{“acc”:[{“y”:-0.718536376953125,”ts”:1481228754,”z”:-0.679046630859375,”x”:-0.0215606689453125},{“y”:-0.6601409912109375,”ts”:1481228756,”z”:-0.739166259765625,”x”:-0.0580596923828125},{“y”:-0.6686553955078125,”ts”:1481228758,”z”:-0.731658935546875,”x”:-0.0715789794921875},{“y”:-0.78422900390625,”ts”:1481228760,”z”:-0.6191864013671875,”x”:0.1313323974609375},{“y”:-0.718658442265625,”ts”:1481228762,”z”:-0.6859588623046875,”x”:0.0318450927734375},{“y”:-0.67626953125,”ts”:1481228763,”z”:-0.7112274169921875,”x”:-0.0835723876953125},{“y”:-0.661224365234375,”ts”:1481228765,”z”:-0.736602783203125,”x”:-0.09320068359375},{“y”:-0.6306610107421875,”ts”:1481228767,”z”:-0.7520294189453125,”x”:-0.1201934814453125},{“y”:-0.3518829345703125,”ts”:1481228769,”z”:-0.966888427734375,”x”:0.10601806640625},{“y”:-0.348968505859375,”ts”:1481228771,”z”:-0.928192138671875,”x”:0.012481689453125},{“y”:-0.4845733642578125,”ts”:1481228773,”z”:-0.844390869140625,”x”:-0.1834869384265625},{“y”:-0.413604236328125,”ts”:1481228775,”z”:-0.8881683349609375,”x”:-0.1864013671875},{“y”:-0.3807373046875,”ts”:1481228777,”z”:-0.8970794677734375,”x”:-0.1714019775390625},{“y”:-0.3825531005859375,”ts”:1481228779,”z”:-0.9042205810546875,”x”:-0.1655731201171875},{“y”:-0.3845062255859375,”ts”:1481228781,”z”:-0.89056396484375,”x”:-0.17230224609375},{“y”:-0.609954833984375,”ts”:1481228783,”z”:-0.8035430908203125,”x”:-0.061492919921875},{“y”:-0.6096343994140625,”ts”:1481228785,”z”:-0.801239013671875,”x”:0.066436767578125},{“y”:-0.195281982421875,”ts”:1481228787,”z”:-1.051605224609375,”x”:-0.262664294921875},{“y”:-0.060882568359375,”ts”:1481228789,”z”:-0.9753570556640625,”x”:-0.5381927490234375},{“y”:-0.42186279296875,”ts”:1481228791,”z”:-0.888916015625,”x”:-0.342015380859375},{“y”:-0.5210418701171875,”ts”:1481228793,”z”:-0.77203369140625,”x”:0.019439697265625},{“y”:0.0989532420703125,”ts”:1481228795,”z”:-0.9192657420703125,”x”:0.0392303466796875},{“y”:0.0200653076171875,”ts”:1481228797,”z”:-0.9849853515625,”x”:0.0487060546875},{“y”:-0.0234222412109375,”ts”:1481228799,”z”:-0.989288330078125,”x”:0.0779876708984375},{“y”:-0.0243988037109375,”ts”:1481228801,”z”:-0.9890899658203125,”x”:0.0768585205078125},{“y”:-0.0255279541015625,”ts”:1481228803,”z”:-0.988128662109375,”x”:0.0744281494140625},{“y”:-0.0251922607421875,”ts”:1481228805,”z”:-0.99005126953125,”x”:0.076873779296875},{“y”:-0.02459716796875,”ts”:1481228807,”z”:-0.989959716796875,”x”:0.075439453125},{“y”:-0.02581787109375,”ts”:1481228809,”z”:-0.9901885986328125,”x”:0.0762939453125},{“y”:-0.025726318359375,”ts”:1481228811,”z”:-0.9894256591796875,”x”:0.0765838623046875},{“y”:-0.0249786376953125,”ts”:1481228813,”z”:-0.9901580810546875,”x”:0.0771026611328125}],”gyro”:[{“y”:-0.1444378733009925,”ts”:1481228754,”z”:0.08388744380862394,”x”:-0.1279257419102832},{“y”:-0.0001358212155940412,”ts”:1481228756,”z”:0.01020629856161967,”x”:-0.02998879177094627},{“y”:0.00420992505119726,”ts”:1481228758,”z”:0.004120442838570598,”x”:-0.00859561873433975},{“y”:-0.07205315487263883,”ts”:1481228760,”z”:0.08386507325546727,”x”:-0.1131318820548931},{“y”:-0.0255330569511342,”ts”:1481228762,”z”:-0.01642484213544818,”x”:-0.03587783988943849},{“y”:0.0253988336321942,”ts”:1481228764,”z”:0.00238459444005695,”x”:-0.04692995841326733},{“y”:-0.009640110513868826,”ts”:1481228766,”z”:0.0007371629897339331,”x”:-0.02342223542113789},{“y”:0.6565626856587939,”ts”:1481228768,”z”:0.928867445009979,”x”:-0.37266092502425},{“y”:0.02560363072002179,”ts”:1481228770,”z”:0.0003398193550941108,”x”:0.02763775316062432},{“y”:0.2570602926393534,”ts”:1481228772,”z”:0.7070160061942362,”x”:0.5762233713704826},{“y”:0.03710662241640105,”ts”:1481228774,”z”:-0.00757189961131329,”x”:-0.06700699723915669},{“y”:0.06019729433182407,”ts”:1481228776,”z”:0.05888781802383212,”x”:-0.04263427957496951},{“y”:0.007393467818277981,”ts”:1481228778,”z”:0.01179194467465285,”x”:0.00510900823520801},{“y”:0.01711800053870232,”ts”:1481228780,”z”:0.001634648277090637,”x”:0.04040574637429121},{“y”:0.0212578844532305,”ts”:1481228782,”z”:0.006425675078143186,”x”:0.006244580124017797},{“y”:0.8008759230207708,”ts”:1481228784,”z”:-0.07184889041702976,”x”:-0.2555540087268046},{“y”:-0.04323375713634636,”ts”:1481228786,”z”:0.1506326523126258,”x”:0.06987495541706303},{“y”:1.133985976380645,”ts”:1481228788,”z”:1.891157561700452,”x”:0.4053813211257905},{“y”:0.07500819841819076,”ts”:1481228790,”z”:0.06159944864575079,”x”:0.3416721162644289},{“y”:-1.563946410155104,”ts”:1481228792,”z”:-1.289318841701152,”x”:-0.7781623542157031},{“y”:-0.2406906403669634,”ts”:1481228794,”z”:0.01702079515891443,”x”:-0.05741402467658226},{“y”:0.246493402066137,”ts”:1481228796,”z”:-0.176484489646243,”x”:0.3039817289371325},{“y”:0.04037298949338324,”ts”:1481228798,”z”:0.003927629975648861,”x”:-0.01290780917139605},{“y”:0.01271978999843646,”ts”:1481228800,”z”:0.006290386494267161,”x”:-0.004387291579796537},{“y”:0.009534116702483672,”ts”:1481228802,”z”:0.005234209438659751,”x”:-0.004364122078312842},{“y”:0.01058313585586588,”ts”:1481228804,”z”:0.007348194079746634,”x”:-0.00546693708571466},{“y”:0.004220577695557576,”ts”:1481228806,”z”:0.004186489233604563,”x”:-0.004337756783521063},{“y”:0.007415039423107623,”ts”:1481228808,”z”:0.004191815555784221,”x”:-0.003278084985778534},{“y”:0.008458732254309676,”ts”:1481228810,”z”:0.006272276999354622,”x”:-0.00650983096858969},{“y”:0.01059006007420009,”ts”:1481228812,”z”:0.005181446216858166,”x”:-0.007559116438080909}],”msgType”:2,”userId”:”7FC35737-7E17-44B2-B568-34243E5CFFE2”,”gps”:{“lat”:42.59748855522223,”lng”:7.612358923694665,”ts”:1481228814}}]

Security

Security is an important aspect of an IoT solution. We want to have only authorized devices sending data to the IoT hub and through an encrypted channel.

We plan to use the Azure IoT Hub access security functionality. This functionality grants access to endpoints by verifying a token against the shared access policies and credentials stored in the IoT hub’s identity registry. Basically, every device is uniquely authorized to send data to our endpoint.

For securing the communication, we send the data over HTTPS in order to avoid the data being read or changed by non-authorized people.

SDKs used, languages, and more

We started by first implementing the Android version because the application prepared by Chip-ing was in a more mature state.

Thanks to a code sample, we were able to serialize the data as a JSON message and send it to the IoT hub that we have prepared in less than one hour.

Once the round trip was working on Android, we also implemented the iOS version.

The following is an example of the device UI in action:

Device UI

Artifacts

Setup of the IoT hub:

Setup of IoT hub

Configuration of Stream Analytics, picking up data from the IoT hub as it arrives, sending it to four different SQL tables

For our prototype, we used Azure SQL Database as our data storage for the sake of simplicity. For a production environment, a NoSQL database might be more convenient and scalable. A more detailed analysis on which storage mechanism—or combination thereof—will be made at a later stage.

We decided to use a single point of entry for all the message types received by the IoT hub. All four messages are hierarchical in nature. Because we want to optimize the payload, we don’t repeat static data such as User ID, Ball ID, and so on.

This required us to implement a more complex Stream Analytics query to split the data and to normalize it for storage in the SQL database.

First, we need to detect the message type—that is, which message is sent. We do this through a type system. Unfortunately, Stream Analytics cannot query the message header from IoT hub, only the JSON payload itself. This requires us to include the message type as part of the message. Therefore, we used the msgType to distinguish them.

The next step is to use the “CROSS APPLY” statement to be able to normalize the hierarchical JSON data to be stored in a SQL table.

SQL Table

This is the actual SQL code used in Stream Analytics.

SELECT
	e.userId AS userid,
	d.ArrayValue.id AS ballid, 
	d.ArrayValue.pwr AS txPower, 
	d.ArrayValue.cal AS calibration,
	d.ArrayValue.lat AS latitude,
	d.ArrayValue.lng AS longitude,
	d.ArrayValue.ts AS rssiTimestamp,
	d.ArrayValue.rssi AS rssi
INTO
	[chipingSQLRSSIValues]
FROM
	[chipingIoTHubSrc] AS e
CROSS APPLY GetElements(e.rssiData) AS d
WHERE e.msgType = 1

SELECT
	e.userId AS userid,
	a.ArrayValue.ts AS timestamp, 
	a.ArrayValue.x AS x, 
	a.ArrayValue.y AS y,
	a.ArrayValue.z AS z
INTO
   		[chipingSQLAcceleration]
FROM
	[chipingIoTHubSrc] AS e
CROSS APPLY GetElements(e.acc) AS a
WHERE e.msgType = 2

SELECT
	e.userId AS userid,
	g.ArrayValue.ts AS timestamp, 
	g.ArrayValue.x AS x, 
	g.ArrayValue.y AS y,
	g.ArrayValue.z AS z
INTO
	[chipingSQLGyro]
FROM
	[chipingIoTHubSrc] AS e
CROSS APPLY GetElements(e.gyro) AS g
WHERE e.msgType = 2

SELECT
	e.userId AS userid,
	h.ArrayValue.ts AS timestamp, 
	h.ArrayValue.x AS x, 
	h.ArrayValue.y AS y,
	h.ArrayValue.z AS z
INTO
	[chipingSQLHeading]
FROM
	[chipingIoTHubSrc] AS e
CROSS APPLY GetElements(e.heading) AS h
WHERE e.msgType = 2

Once we had the data flowing from the ball to the SQL database, we used Power BI to quickly visualize it through charts and maps.

The following image shows Power BI in action, displaying some early data.

Power BI

Pointers to references or documentation

Stream Analytics: cross apply

Stream Analytics: msgType

Learnings from the Microsoft team and the customer team

One really helpful component is Azure’s Activity Log, in this case seen in Stream Analytics. It allowed us to see when something goes wrong with the data being moved from the IoT hub to the database. In the case shown below, you can see a problem transforming the data from JSON to SQL.

Problem

One simple yet important learning is to click on the JSON tab in order to see a more detailed error description. In this case, the SQL field length was shorter than the data provided by JSON, making the Stream Analytics job fail.

Job fail

Another really great feature is the Data Sampling in Stream Analytics that allows you to see the exact message picked up from the IoT bub at design time. Unfortunately, the current version doesn’t allow multiple queries to be run against the sample data, so we had to test our four queries independently.

Queries

Conclusion

In the few days of the hackfest, we were able to create a first, valid proof of concept that really worked. It’s easy to imagine how to take this solution to the next level. The combination of IoT Hub, Stream Analytics, SQL, and Power BI proved to be a very simple yet powerful way to tackle our problem with rapid prototyping. Having a platform that allows you to easily scale to thousands of customers using tens of thousands of golf balls becomes feasible even for a small startup. The management and diagnostic tools provided by Azure are complete and still easy enough to use, allowing even a small team to run a highly scalable production environment.

Opportunities going forward

In case multiple players using Chip-ing’s solution play together, we want to be able to triangulate the position of a lost ball in play through multiple phones by aggregating their data. We think we could use the IoT Hub bidirectional communication to synchronize the data among the players’ devices.

We are currently collecting real data from players using the prototype balls and solution/app for their golf plays. This allows us to experiment with machine learning, trying first to improve the location algorithm, helping players to find their lost golf balls even quicker. Further down the road, we want to become able to predict player habits by comparing their play data to existing, collected data from players of varying skills. This will allow us to provide them with tips on improving their game just by analyzing how they are playing.

Machine learning idea for signal improvements:

ML idea

First machine learning experiment:

ML experiment

We might also explore the use of Power BI Embedded to create a player-friendly dashboard showing statistics about their plays, current and historical.