Virtual Assistant Client (Android)

Chat with your Virtual Assistant using the Virtual Assistant Client app and set it up as the default assistant on a device.

Prerequisites

1. Install Android Studio.

2. Download the Virtual Assistant Client app source code.

3. Create a Virtual Assistant to setup your Virtual Assistant environment.

4. Enable speech on your new Virtual Assistant

5. If you want to capture analytics, get started with Visual Studio App Center and register a new app.

6. If you want to enable authentication, get started with the Linked Accounts solution accelerator.

Build and run

Add your application settings

There are two configuration files used to provide your environment settings.

Direct Line Speech configuration

{
  "SpeechSubscriptionKey": "SPEECH_SERVICE_SUBSCRIPTION_KEY", // Replace with your Speech Service subscription key
  "SpeechRegion": "westus2",
  "CustomCommandsAppId": "", // Optional, if you are connecting to a Custom Commands application
  "CustomVoiceDeploymentIds": "", // Optional, to point to custom voices
  "CustomSREndpointId": "", // Optional, to point to a customized speech recognition endpoint
  "TTSBargeInSupported": false, // Whether or not to listen to keyword while TTS is playing. If true, TTS playback stops once Keyword is verified.
  "SpeechSDKLogEnabled": false, // Optional, whether to log Direct Line Speech activites. The default file path is on the device internal storage at Android/data/com.microsoft.bot.builder.solutions.virtualassistant/files/SpeechSDK.log
  "UserId": "android",
  "UserName": "Android",
  "SRLanguage": "en-us",
  "Keyword": "computer",
  "EnableKWS": false, // Keyword spotting
  "LinkedAccountEndpoint": "" // Optional if you enabled the Linked Accounts solution in the prerequisites
}

The UserId is a unique identifier for all messages generated by the user, this can be combined with Linked Accounts sample.

App configuration

{
  "history_linecount": 2147483646,
  "show_full_conversation": true,
  "enable_dark_mode": false,
  "keep_screen_on": true,
  "app_center_id": "APP_CENTER_ID" // Replace with your Visual Studio App Center id
}

Optional: Chat colors

<?xml version="1.0" encoding="utf-8"?>
<resources>
...
    <!-- Chat -->
    <color name="color_chat_text_bot">#000000</color>
    <color name="color_chat_text_user">#ffffff</color>
    <color name="color_chat_background_bot">#f2f2f2</color>
    <color name="color_chat_background_user">#3062d6</color>
...
</resources>

Run

Build and run your app to deploy to the Android Emulator or a connected device.

Permissions

Record Audio

Required for the user to make voice requests to a bot. With this a user can only use the keyboard.

Fine Location

Allow Virtual Assistant to receive the VA.Location event with GPS coordinates to utilize location-based skills like Point of Interest.

Interact with a Virtual Assistant

Chat

The main view shows an expected user and assistant chat window. Start a conversation by selecting the microphone or keyboard icons.

Widget

Using widgets, you can demonstrate an Assistant having a native chat experience on a device.

Side menu

Swipe from the left to access the menu.

Sign in

If you enabled the Linked Accounts solution, the optional Sign in option will appear.

Restart conversation

Restart the conversation with a Virtual Assistant with a new conversation id.

Settings

Access the same settings from the configuration files.

Set as default assistant

Set your Virtual Assistant as the device’s default assist app.

1. Allow the Appear on top permission to overlay this app on Android

2. Select Device assistance app

3. Select Virtual Assistant

Authenticating users

If you provided a Linked Accounts solution endpoint in the app’s configuration settings, a Sign in option will be available on the side menu.

1. Select Sign in and authenticate with the Linked Accounts app.

2. The user id that will be used in each outgoing Activity is highglighted at the top, next to a Sign out all button that will unlink all of your accounts with a bot. There is a row for each authentication provider enabled on your bot, select Link to sign in with that specific account.

3. After linking an account, the button will change to Unlink. SelecT Complete and return to app to pass the user id back to the Android app.

4. Return to the app and the side menu now shows Sign out. Any future conversations with your Virtual Assistant will now pass the linked user id.

Events

The Virtual Assistant Client is enabled to work with events used in the sample Skills.

Open default apps

OpenDefaultApp

This method takes the metadata from an OpenDefaultApp event to open default apps on the device.

Maps

Compatible with either Waze or Google Maps (in this order).

Phone

Compatible with the default dialer.

Music

Compatible with Spotify.

Other events

BroadcastWidgetUpdate

This method sends the value of this event activity to any listening apps, like the Event Companion app.

Next steps

Use the Event Companion app to broadcast your Virtual Assistant’s metadata and prototype advanced scenarios.