Microsoft teamed up with Gump Come, a provider of smart vending machines, to develop a way to monitor the working status of its machines and to get real-time information on replenishment needs.

Key technologies used

Core team

  • Wenbin Cao – CTO, Gump Come
  • Hongjie Sun – CMO, Gump Come
  • YunLong Dong – Chief Architect and IT Pro Manager, Gump Come
  • Peng Liu – Software Engineer, Gump Come
  • Yan Zhang – Audience Evangelism Manager, Microsoft
  • Shijun Liu – Technical Evangelist, Microsoft
  • Leon Liang – Technical Evangelist, Microsoft
  • Yuheng Ding – Technical Evangelist, Microsoft
  • Zepeng She – Technical Evangelist, Microsoft
  • Lit Li – Technical Evangelist, Microsoft

Special thanks to the Gump Come team, the Microsoft China DX Technical Evangelist team, and the Microsoft Audience Evangelism team.


Figure 1. The Gump Come smart vending machine deployment in Microsoft Technical Center

Gump Come smart vending machine deployment in Microsoft Technical Center


Customer profile

Gump Come is a leading AI innovation-oriented provider of smart vending machines and smart micro-supermarkets in China. These machines integrate the use of cameras, multi-touch, voice interaction, and big data analysis to realize new human-computer interaction patterns.

Problem statement

To date, Gump Come has no solution that can monitor vending machine health status or get real-time information on whether a device is working. Any device needing attention usually requires two onsite visits—once to diagnose the issue and again to return with the component to fix the issue. All this adds to the cost of maintaining the machines.

The Gump Come goods manager must check on the selling status of the vending machines to see if they need replenishing. They want to develop an app for this. The goods manager will need training to learn to use the app. They want to get this information only through text or voice on the web app, and to reduce their cost of installing, learning, and updating the application.

Solution and steps

The Smart Vending Machine solution will do the following:

  1. Use Azure IoT Hub to collect data on supplies and device health for all vending machines and send it to the uniform management platform for daily monitoring. This will help the staff to diagnose device issues online, which can save on maintenance costs and help realize predictive maintenance cost savings.

  2. Build a vending machine management bot, based on Bot Framework and LUIS, for an operator to check on vending machines and the status of their goods supply.

In this Smart Vending Machine solution, Gump Come will use the following Microsoft technologies:

  • IoT Hub
  • Stream Analytics
  • SQL Database
  • Power BI
  • Bot Framework

Architecture

The Gump Come Smart Vending Machine solution architecture can be represented as follows:

  • The Gump Come vending machine client will send its device status data to the Azure IoT Hub every 3 minutes. This data includes device status, goods channel status, and temperature.

  • Stream Analytics will transfer this data into three tables on Azure SQL Database, and then Power BI will synch these table updates into the admin dashboard.

  • The Gump Come bot will offer voice and text query features to the operator to check the vending machine device status and goods channel status.


Figure 2. Gump Come Smart Vending Machine solution architecture

Gump Come Smart Vending Machine solution architecture


Note: Power BI is not generally available in China, so Stream Analytics can’t configure Power BI as an output stream in this phase.


Technical delivery

Device used and code artifacts

Microsoft’s China DX Technical Evangelist team and the Gump Come dev team split the engagement into four segments:

Vending machine IoT SDK integration

In this segment, the Gump Come team focused on vending machine client Azure IoT Hub SDK integration. The Gump Come vending machine client is the Android OS. The Microsoft DX team helped the Gump Come client dev team to code the Azure IoT Hub Android SDK in their client, set up and register the vending machine client with Azure IoT Hub, connect with Azure IoT Hub, and send vending machine device status data to Azure IoT Hub. The Android client code screenshot is as follows:

package com.gumpcome.iot;
import android.os.SystemClock;

import com.gumpcome.kernel.debug.GumpLog;
import com.microsoft.azure.iothub.IotHubEventCallback;
import com.microsoft.azure.iothub.IotHubStatusCode;
import com.microsoft.azure.iothub.DeviceClient;
import com.microsoft.azure.iothub.IotHubClientProtocol;
import com.microsoft.azure.iothub.IotHubEventCallback;
import com.microsoft.azure.iothub.IotHubMessageResult;
import com.microsoft.azure.iothub.IotHubStatusCode;
import com.microsoft.azure.iothub.Message;
import com.microsoft.azure.iothub.MessageCallback;

import java.io.IOException;
import java.net.URISyntaxException;

/**
 * @author LiuPeng @GumpCome
 *         LiuShijun @ Microsoft
 *         Created at 2016/8/30.
 */
public class IotHelper {
    private static final String TAG = "IotHelper";
    String connString = "HostName=iotxxx.azure-devices.cn;DeviceId=myFirstJavaDevice;SharedAccessKey=od1/RiE0ItHtbuewQ/xxxxxxxx";

    public void SendMessage(String message) throws URISyntaxException, IOException {
        IotHubClientProtocol protocol = IotHubClientProtocol.AMQPS;
        DeviceClient client = new DeviceClient(connString, protocol);

        try {
            client.open();
        } catch (IOException e) {
            GumpLog.e(TAG, "Iot连接异常:" + e.toString());
        }
        try {
            Message msg = new Message(message);
            msg.setProperty("messageCount", Integer.toString(1));
            System.out.println(message);
            EventCallback eventCallback = new EventCallback();
            client.sendEventAsync(msg, eventCallback, 1);
        } catch (Exception e) {
        }
        SystemClock.sleep(2000);
        client.close();
    }

    protected static class EventCallback implements IotHubEventCallback {
        public void execute(IotHubStatusCode status, Object context) {
            Integer i = (Integer) context;
            GumpLog.d(TAG, "IoTHub 返回的消息 " + i.toString()
                    + " ,状态:" + status.name());
        }
    }

    public void receiveMessage() throws URISyntaxException, IOException {

        IotHubClientProtocol protocol = IotHubClientProtocol.AMQPS;

        DeviceClient client = new DeviceClient(connString, protocol);
        {
            MessageCallback callback = new MessageCallback();
            Counter counter = new Counter(0);
            client.setMessageCallback(callback, counter);
        }
        try {
            client.open();
        } catch (IOException e) {
            GumpLog.e(TAG, "Iot连接异常:" + e.toString());
        }
        SystemClock.sleep(2000);
        client.close();
    }

    protected static class MessageCallback implements com.microsoft.azure.iothub.MessageCallback {
        public IotHubMessageResult execute(Message msg, Object context) {
            Counter counter = (Counter) context;

            GumpLog.d(TAG,
                    "Iot收到的消息:" + counter.toString()
                            + ",内容: " + new String(msg.getBytes(), Message.DEFAULT_IOTHUB_MESSAGE_CHARSET));

            int switchVal = counter.get() % 3;
            IotHubMessageResult res;
            switch (switchVal) {
                case 0:
                    res = IotHubMessageResult.COMPLETE;
                    break;
                case 1:
                    res = IotHubMessageResult.ABANDON;
                    break;
                case 2:
                    res = IotHubMessageResult.REJECT;
                    break;
                default:
                    throw new IllegalStateException("Invalid message result specified.");
            }

            GumpLog.d(TAG, "Responding to message " + counter.toString() + " with " + res.name());

            counter.increment();

            return res;
        }
    }

    /**
     * 作为消息的回调
     */
    protected static class Counter {
        protected int num;

        public Counter(int num) {
            this.num = num;
        }

        public int get() {
            return this.num;
        }

        public void increment() {
            this.num++;
        }

        @Override
        public String toString() {
            return Integer.toString(this.num);
        }
    }
}


Vending machine Azure IoT Hub, Stream Analytics, and Azure SQL Database deployment

In this segment, the Gump Come team deployed IoT Hub, Stream Analytics, and SQL Database on Azure. First, as a Gump Come proof of concept (POC) for Azure IoT services, Gump Come created one free IoT hub in their subscription, deployed Stream Analytics, and created their device status database on SQL Database. Next, Gump Come technical decision makers, based on this POC result, plan to release this IoT solution to their 2,000+ production devices in China.


Figure 3. Azure IoT Hub configuration in Azure

Azure IoT Hub Configuration in Azure


Figure 4. SA input streaming configuration in Azure

SA input streaming configuration in Azure


Figure 5. SA output streaming configuration in Azure

SA output streaming configuration in Azure


Data visualization in Power BI

The Gump Come team presented their device status data and goods channel data through Power BI. They liked Power BI’s excellent graphics and ease of use for the business operator. But because Power BI isn’t generally available in China, we couldn’t connect Stream Analytics directly with Power BI. Instead, the Gump Come team stored the data from Stream Analytics to Azure SQL Database, and used Power BI to connect to SQL Database.


Figure 6. Gump Come vending machine status data in Power BI

Gump Come vending machine status data in Power BI


Vending machine administrator bot dev

Here, the Gump Come team developed the vending machine management bot, which allows the vending machine operator to check the vending machine device status and goods status through a mobile phone.


Figure 7. Vending machine management bot

Vending machine management bot


Conclusion and opportunities going forward

As a result of this technical engagement, the Gump Come technical and business decision makers plan to roll out this IoT vending machine solution to all 2,000+ devices in China by the end of 2016, hosted entirely on Microsoft Azure.

The Gump Come team will integrate the vending machine management bot app with WeChat in the coming months.

Additional resources


Figure 8. The Gump Come hackfest team

Gump Come hackfest team