Table of Contents

Add Cortana Support

At this point we have a pretty cool event app that is sure to impress our invitees - but why don't we really wow them with voice integration? Setting up Cortana is a simple step that will really make our app awesome.

Download Code

Set Up Cortana

1

To begin, we need to create a set of rules for Cortana. The rules will explain what she should listen for and what various phrases mean, and how the device should react to commands.

We’ll start by adding a simple XML file. Once your app is deployed, this file will explain to Cortana what she should listen for. I’m naming it “CortanaCommands.xml”.

Here’s the start to the file, you can just copy and paste this into your new XML file:

<?xml version="1.0" encoding="utf-8"?>
    <VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.1">
    <CommandSet xml:lang="en-us" Name="UWPQuickStart_en-us">
        <CommandPrefix> Event, </CommandPrefix>
        <Example> Event Register </Example>
        
        <Command Name="GetStarted">
            <Example> Register  </Example>
            <ListenFor> Register </ListenFor>
            <Feedback>  Let’s check out the event </Feedback>
            <Navigate/>
        </Command>

    </CommandSet>
    <!-- Other CommandSets for other languages -->
    </VoiceCommands>
2

You can see that I’ve included a CommandPrefix of “Event”. This means that Cortana will now listen for the keyword "Event" in any commands she gets. If she hears that keyword, she’ll take the rest of the phrase and see if it matches any of the “Listen For” phrases. If it does, she’ll launch our app. Not only that, but she’ll pass the command to the app to trigger some app functionality. We’ll play with that later. The Feedback tag is what Cortana will say to the user when she recognizes the command.

Now we need to register the commands with Cortana. That’s easy enough to do in my OnLaunched event in my App.xaml.cs class. This does mean that you’ll have to launch the app at least once before Cortana will work, but that’s OK for now.

Here are the commands you need to add to the top of the OnLaunched event in App.xaml.cs:

var storageFile = await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///CortanaCommands.xml"));
    await VoiceCommandDefinitionManager.InstallCommandDefinitionsFromStorageFileAsync(storageFile);

Tip: You’ll need to make the event handler async. You’ll also need to add a using statemtn for the VoiceCommandDefinitionManager library - It’s in Windows.ApplicationModel.VoiceCommands

3

We can already test it out. Launch the app first to make sure we’ve deployed the Cortana commands, then close it. Next, start up Cortana either by clicking the microphone next to her, or if it’s enabled, say “Hey Cortana”. When Cortana is listening, say “Event Register". If everything is set up correctly, Cortana should launch the event.

Add Voice Commands

1

That’s pretty cool already, but let’s go another step. If I’m at my event and I want to see the photos, maybe I don’t want to have to navigate. Let’s have Cortana jump right to the photos page for me.

First, I need to add a new command, so here’s the code I’ve added to CortanaCommands.xml: 

<Command Name="Photos">
    <Example> Photos  </Example>
    <ListenFor> Photos </ListenFor>
    <Feedback>  Let’s look at some pictures. </Feedback>
    <Navigate/>
</Command>
2

We’ve now got two commands, GetStarted and Photos. Unfortunately, our app doesn’t know what to do if Cortana passes either of these commands to us. It’s just going to launch and let us deal with it.

Whenever an app starts, either the OnLaunched or OnActivated event will be fired. OnLaunched will fire if the app is launched “normally”. This includes clicking on the tile or if Cortana launches it without a command. However, if Cortana launches the app with a command, like the way we’ve been doing it, the OnActivated event will fire instead.

The default template has all of the initialization code in OnLaunched, but we need that same code to execute when we’re activated instead. So, we need to refactor that code. To save ourselves a bit of complication in the refactor, locate the line with the call to rootFrame.Navigate(typeof(EventMainPage), e). Remove the e parameter. The chunk of code should now look like this: 

if (rootFrame.Content == null)
{
    // When the navigation stack isn't restored navigate to the first page,
    // configuring the new page by passing required information as a navigation
    // parameter
    rootFrame.Navigate(typeof(EventMainPage));
}
3

Now we need to select the contents of the OnLaunched handler, starting with the two lines we added to register the voice commands through the call to Window.Current.Activate(). With those lines selected, choose Edit / Refactor / Extract Method. This will move all of this code to its own method. Now we can call into that from our OnActivated event handler.

Your code should look like this now: 

protected async override void OnLaunched(LaunchActivatedEventArgs e)
{
    await InitializeFrame();
}

private async System.Threading.Tasks.Task InitializeFrame()
{
    …
}
4

Great, now we’ve got what we need to create our OnActivated event handler. Put your cursor on a line inside the App.xaml.cs that isn’t in a method (but is inside the class). Start typing “protected override void” and Intellisense should come up with a list of possible event handlers. OnActivated should be in the list (for me, it’s at the top). Just hit enter and it’ll put the stub into your code for you. 

protected override void OnActivated(IActivatedEventArgs e)
{
    base.OnActivated(e);
}

We need to call the method to initialize our Frame, so just add that call. You’ll also need to make this event handler async. 

protected async override void OnActivated(IActivatedEventArgs e)
{
    await InitializeFrame();
    base.OnActivated(e);
}

Hooking up Navigation on Launch

1

At this point, I realize that the event as it exists doesn’t have the ability to navigate to a page at launch other than the start page. Well, that’s not what we had in mind, but we will extend that now.

Open up EventMainPage.xaml.cs. Near the bottom is an event handler for NavMenu_ItemClickHandler. That handler does three things, one of which is to actually navigate. We need to extract that line to its own method. So, highlight that line and again choose Edit / Refactor / ExtractMethod. I called it SwitchToWindow. You may need to tweak it slightly, but we want our method to take an argument of type “Type”. We’ll use this to navigate to the page we want later.

Your code should look like this:

private void NavMenu_ItemClickHandler(object sender, ItemClickEventArgs e)
{
    var destPage = (e.ClickedItem as NavMenuItem)?.DestPage;
    SwitchToWindow(destPage);
    rootSplitView.IsPaneOpen = false;
}

public void SwitchToWindow(System.Type desiredPage)
{
    AppNavigationUtil.SetSplitViewContent(rootSplitView, desiredPage, true);
}

Now we’re ready to finish our OnActivated event handler. By the time we’re done, we want to do three things. In order: we want to figure out which command Cortana gave us, Initialize our Frame, and navigate to the Photos page if we got the “Photo” command.

2

When our app is started with a voice command, the args passed to us are actually of the type VoiceCommandActivatedEventArgs. Once we know that we were activated with a voice command, we can cast the args to the right type. We can figure out how we were activated with the “Kind” property which comes from the IActivatedEventArgs interface.

Put all this together and your code will end up looking like this:

protected async override void OnActivated(IActivatedEventArgs args)
{
    base.OnActivated(args);

    if (args.Kind != ActivationKind.VoiceCommand)
    {
        return;
    }

    var EventArgs = args as VoiceCommandActivatedEventArgs;
    SpeechRecognitionResult ActivateResult = EventArgs.Result;
    string CommandName = ActivateResult.RulePath[0];
}
3

When our app is started with a voice command, the args passed to us are actually of the type VoiceCommandActivatedEventArgs. Once we know that we were activated with a voice command, we can cast the args to the right type. We can figure out how we were activated with the “Kind” property which comes from the IActivatedEventArgs interface.

Put all this together and your code will end up looking like this:

protected async override void OnActivated(IActivatedEventArgs args)
{
    base.OnActivated(args);

    if (args.Kind != ActivationKind.VoiceCommand)
    {
        return;
    }

    var EventArgs = args as VoiceCommandActivatedEventArgs;
    SpeechRecognitionResult ActivateResult = EventArgs.Result;
    string CommandName = ActivateResult.RulePath[0];
}
4

We’ve got the CommandName now and we can just switch on that. Once we know which command we’re using, we pass the right type to SwitchToWindow and we’re golden. The switch statement is pretty simple…

switch (CommandName)
{
    case "GetStarted":
        {
            // We don't want to do anything special in this case
            break;
        }
    case "Photos":
        {
            break;
        }
    default:
        {
            // We got a command we don't recognize, but for now we'll just ignore
            break;
        }
}
5

Now we want to call SwitchToWindow with the right type. We're going to assume we want to go to the EventHome unless Cortana hear's otherwise. So, just before the switch, I’m going to initialize a variable with typeof(EventHome) as a value and change it when I need to (when Cortana hears "Photos"). Now my code looks like this:

Type StartingPage = typeof(Views.EventHome);

switch (CommandName)
{
    case "GetStarted":
        {
            // We don't want to do anything special in this case
            break;
        }
    case "Photos":
        {
            StartingPage = typeof(Views.Photos);
            break;
        }
    default:
        {
            // We got a command we don't recognize, but for now we'll just ignore
            break;
        }
}
6

Let’s pass that StartingPage to SwitchToWindow and we’re done. Unfortunately, SwitchToWindow is on the rootFrame. We can get that the long way, but since our InitializeFrame method uses the rootFrame, let’s just return it. All I need to do is modify the signature of the InitializeFrame method to return the frame, and I can use that.

Change the InitializeFrame method signature to this:

private async System.Threading.Tasks.Task<Frame> InitializeFrame()

Notice the <Frame> at the end of the return type. Now at the bottom, you can add a simple “return rootFrame” to get that object out.

7

Back to our OnActivated event handler, let’s call InitializeFrame and capture the Frame on the way out. Add these lines after the switch:

Frame rootFrame = await InitializeFrame();
(rootFrame.Content as EventMainPage).SwitchToWindow(StartingPage);

The first line grabs the rootFrame and the second line casts its Content as our EventMainPage. Once we’ve got that, we can call SwitchToWindow and we’re in business.

8

Since this has been a bit disjointed, here are the key methods that we’ve updated so far:

/// <summary>
///     Invoked when the application is launched normally by the end user.  Other entry points
///     will be used such as when the application is launched to open a specific file.
/// </summary>
/// <param name="e">Details about the launch request and process.</param>
protected async override void OnLaunched(LaunchActivatedEventArgs e)
{
#if DEBUG
    if (Debugger.IsAttached)
    {
        DebugSettings.EnableFrameRateCounter = false;
    }
#endif
    await InitializeFrame();
}

private async System.Threading.Tasks.Task<Frame> InitializeFrame()
{
    var storageFile = await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///CortanaRules.xml"));
    await VoiceCommandDefinitionManager.InstallCommandDefinitionsFromStorageFileAsync(storageFile);

    var rootFrame = Window.Current.Content as Frame;

    // Do not repeat app initialization when the Window already has content,
    // just ensure that the window is active
    if (rootFrame == null)
    {
        // Create a Frame to act as the navigation context and navigate to the first page
        rootFrame = new Frame();

        rootFrame.NavigationFailed += OnNavigationFailed;

        // Place the frame in the current Window
        Window.Current.Content = rootFrame;
    }

    if (rootFrame.Content == null)
    {
        // When the navigation stack isn't restored navigate to the first page,
        // configuring the new page by passing required information as a navigation
        // parameter
        rootFrame.Navigate(typeof(EventMainPage));
    }
    // Ensure the current window is active
    Window.Current.Activate();
    return rootFrame;
}

protected async override void OnActivated(IActivatedEventArgs args)
{
    base.OnActivated(args);

    if (args.Kind != ActivationKind.VoiceCommand)
    {
        return;
    }

    var EventArgs = args as VoiceCommandActivatedEventArgs;
    SpeechRecognitionResult ActivateResult = EventArgs.Result;
    string CommandName = ActivateResult.RulePath[0];
    string textSpoken = ActivateResult.Text;
    Type StartingPage = typeof(Views.EventHome);

    switch (CommandName)
    {
        case "GetStarted":
            {
                // We don't want to do anything special in this case
                break;
            }
        case "Photos":
            {
                StartingPage = typeof(Views.Photos);
                break;
            }
        default:
            {
                // We got a command we don't recognize, but for now we'll just ignore
                break;
            }
    }

    Frame rootFrame = await InitializeFrame();
    (rootFrame.Content as EventMainPage).SwitchToWindow(StartingPage);
}
9

With all of this, let’s run our app and get everything registered correctly. Press F5 (or Play) and the app will run. If you set a breakpoint in OnLaunched, you can watch everything get initialized. When I started doing this, I ran into some glitches, and every time I hit Play from Visual Studio, I never got the OnActivated event firing. Well, that’s because a launch from VS is considered “Standard”, so that event never triggers.

Since I wanted to debug the Activated experience, I didn’t know how to tell VS to launch the app as if from Cortana. Turns out that VS has me covered. Right click the project and pick the properties. Under “Debug”, I can choose an option to “Not launch, but debug on launch”. Click that on and hit Play.

Now everything is deployed, but my app hasn’t started. Put a breakpoint in OnActivated so we can see everything do what it’s supposed to. Click on Cortana and say “Event Register”. Your app launches and the debugger kicks in. If everything is set up right, you’ll hit that breakpoint and now you can debug.

Awesome. Close the app and you’ll see that VS doesn’t stop. Click Cortana and say “Event photos”. There we go.