In this project, you learn how to use DevKit as a translator. The app records your voice and translates it to English text shown on the DevKit screen.

What you need

Finish the Getting Started Guide to:

  • Have your DevKit connected to Wi-Fi
  • Prepare the development environment

An active Azure subscription. If you do not have one, you can register via one of the methods:

Step 1. Open the project folder

A. Start VS Code

  • Make sure your DevKit is not connected to your PC.
  • Start VS Code
  • Connect the DevKit to your computer.

VS Code automatically finds the DevKit and opens an introduction page:

Introduction page

Notice: Occasionally, when you launch VS Code, you may be prompted with error saying that the Arduino IDE or related board package cannot be found. If this happens, close VS Code, launch the Arduino IDE once again, and VS Code should locate the Arduino IDE path correctly.

B. Open the Arduino Examples folder

Expand left side ARDUINO EXAMPLES section, browse to Examples for MXCHIP AZ3166 > AzureIoT, and select DevKitTranslator. This will open a new VS Code window with the DEVKITTRANSLATOR project folder in it.

mini-solution-examples

If you happen to close the pane, you can reopen it. Use Ctrl+Shift+P (macOS: Cmd+Shift+P) to open the command palette, type Arduino, and then find and select Arduino: Examples.

Step 2. Provision Azure services

In the solution window, run the cloud-provisioniong task by typing Ctrl+P (macOS: Cmd+P) and entering task cloud-provision in the text box:

In the VS Code terminal, an interactive command line will guide you through provisioning all required Azure services:

mini-solution-provision-sub

Notice: If the page hangs in the loading status when trying to sign in to Azure, plese check this FAQ steps to resolve this issue.

Step 3. Deploy Azure Functions

Use Ctrl+P (macOS: Cmd+P) to run task cloud-deploy to deploy the Azure Functions code. This process usually takes 2 to 5 minutes to complete:

mini-solution-deploy

After Azure Function deploys successfully, fill in the azure_config.h file with function app name. You can navigate to Azure portal to find it:

mini-solution-function-app

Notice: If the Azure Function does not work properly, please check this FAQ steps to resolve this issue.

Step 4. Build and upload the device code

Windows

  1. Use Ctrl+P to run task device-upload.
  2. The terminal prompts you to enter configuration mode. To do so, hold down button A, then push and release the reset button on the DevKit. The screen displays the DevKit ID and ‘Configuration’.

This sets the connection string that is retrieved from the task cloud-provision step.

VS Code then starst verifying and uploading the Arduino sketch to the DevKit:

device-upload

The DevKit reboots and starts running the code.

Notice: Occasionally, you may get an “Error: AZ3166: Unknown package” eror message. This occurs when the board package index is not refreshed correctly. Check this FAQ steps to resolve this issue.

macOS

  1. Put the DevKit into configuration mode: Hold down button A, then push and release the reset button on the DevKit. The screen displays ‘Configuration’.
  2. Use Cmd+P to run task device-upload.

This sets the connection string that is retrieved from the task cloud-provision step.

VS Code then starts verifying and uploading the Arduino sketch to the DevKit:

device-upload

The DevKit reboots and starts running the code.

Notice: Occasionally, you may get an “Error: AZ3166: Unknown package” error message. This is occurs when the board package index is not refreshed correctly. Check this FAQ steps to resolve this issue.

Test the project

After app initialization, follow the instructions on the DevKit screen. The default source language is Chinese.

To translate Chinese to English:

  • Simply press and hold button B while speaking, then release button B to initiate the translation. Several seconds later, the translation will be shown on the screen.

To select a source language other than Chinese:

  • Press button A to enter setup mode.
  • Press button B to scroll all supported source languages.
  • Press button A to confirm your choice of source language.
  • Press and hold button B while speaking, then release button B to initiate the translation. Several seconds later, the translation will be shown on the screen.
Arduino application initializing DevKit Initialized Choose source language-1 Choose source language-2 Ready to talk Translation
Translate as you go
  • Press button A and B to scroll and select the source language.
  • Press button B to talk, release to send the voice and get the translation text

How it works

mini-solution-voice-to-tweet-diagram

The Arduino sketch records your voice then posts an HTTP request to trigger Azure Functions. Azure Functions calls the cognitive service speech translator API to do the translation. After Azure Functions gets the translation text, it sends a C2D message to the device. Then the translation is displayed on the screen.

Problems and feedback

If you encounter problems, you can refer to FAQs or reach out to us from the channels below.

Updated: