[Azure] Building a Cortana based skilled bot

In my previous post I discussed how to build a bot using a Web Project and deploying it as a DLL to an Azure Function. The Azure Function would then function as the endpoint for your bot which you can make available through channels which include Facebook, Skype (for Business) and many more. Starting at //BUILD this year, Cortana was also included in that list of channels. Cortana of course is a bit of a special channel, since we’ll then be using voice to communicate with our bot. But if you think of it, voice translates to text and text is the input of choice for the bot that we build. So it should be that difficult, right? It’s not!

 

Cortana Skills

The Cortana SDK or development framework or whatever you’d like to call it calls this skills. Basically you’re learning Cortana new skills as you would learn a pet or child something new. The set-up on the Cortana side is simpler than you might think. You simply enable the Cortana channel for your bot, fill in some fields and you’re good to go.

cortana-skillOne of these fields is the invocation name. That’s how Cortana will understand she needs to route a message to our bot. For my home automation sample, I’ll be using “The House” as invocation name, so a command might be “Hey Cortana, tell The House to turn on the lights in the kitchen”. If you haven’t read it yet, take 5 minutes to read Cortana kills + LUIS pre-built domains on how to effortlessly teach you bot these kinds of intents.

Another cool thing you can configure is which data Cortana will supply your bot with. Since Cortana is the digital assistant of the user you will be communicating with, she knows things about this user. These things you can request for so that you can make use of the data in your bot code.

cortana-userprofile

Be aware though: the user does need to consent to Cortana sending this information to the bot before she actually does so.

cortana_permission

Developer mode

Another handy thing is to put Cortana herself into Developer Mode. You can do this by heading over to the Cortana Dashboard (the link is displayed with the Cortana channel itself). Here, under the Debug section there’s the option to enable Developer Mode by simply clicking a checkbox.

 

cortana-debug

After which invoking a command will show you debug info:

cortana-debugon

 

Debugging the bot

For testing and debugging your bot, you get all of the options you normally have for bots (and for Azure Functions by the way). Of course you cannot get the online version of Cortana to call a local instance of your bot, but you can simply pass in the text of the intent you want to test via the bot framework emulator. One feature gap at this moment is the fact that the bot emulator does not allow us to pass in the additional channel data that Cortana can supply (context / user info). What you could do is attach your debugger to an Azure hosted instance of your bot and then debug that specific part.

 

That’s it for now, in my next post I’ll be going in a bit deeper into how this all comes together.

 

[Azure] Building a bot using DLL / WebProject

In previous articles I explained how you can build bots using the Microsoft Bot Framework and the Azure Bot Service. The latter is built on top of Azure Functions, one of my favorite components in Azure. Both the Functions and Bot teams are releasing stuff in a fast pace, but sometimes this leads towards the two not being 100% in sync with each other. This post addresses one of these issues, namely the Bot Service having old-style templates for new instances.

When you create a new Bot Service instance and download the code, you get a solution with .CSX files. These are used in Azure Functions and they still work great. The issue though is that when you load these in Visual Studio and want to debug your code locally, there’s no IntelliSense to go with them. Althought this is on the teams backlog, it’s not there yet and if you’re a VS dev like I am, you probably can’t live without it 🙂

 

.NET Class library as Function App

Two months ago, Donna Malayeri (who’s doing absolutely awesome work on the Functions team) wrote this post detailing how you can build a web project which uses the local functions runtime to host and debug the code. This brings two great worlds together: it allows you to build code in Visual Studio with all the benefits (Intellisense!) whilst utilizing the func.exe CLI runtime as well.

The post does a very good job at explaining how to set this up, but what if you want this for your Bot Service instances?

 

Converting a Bot Service

To convert your bot service instance to a Web Project, here’s what you need:

  • A web project (well duh…).
  • The CSX files that you want to convert to ‘regular’ C# classes.
  • The function.json file that defines the endpoint for your bot.
  • A appsettings file should you have one. This is typically where your Microosft App ID and password are stored.
  • The project.json file. You don’t really need this, but it’s handy to look up which packages your instance is using.

And here’s the steps:

  • Follow Donna’s post to set-up a new web project.
  • When creating the classes, create one for your dialog and one for your entry point. You can combine them as well of course, but I personally like to separate them. In the example these are named Dialog.cs and DialogEntryPoint.cs.
  • DialogEntryPoint.cs will contain the contents from the ‘run.csx’ file. This the entry point that’s being called when the user communicates with your bot. It’s referenced in function.json as follows:

    Note the scriptFile and entryPoint settings which point to the output DLL and the class/method which handles the incoming message.
  • Dialog.cs contains your dialog code.
  • Ensure that you set-up the project to load the correct NuGet packages. Normally you will need Microsoft.Bot.Builder.Azure which will include some dependencies. If you include this, ensure the project is set-up to run .NET framework version 4.6 instead of 4.5.

Lastly, you need to configure the start options of the project. This is detailed in Donna’s post as well, but for your reference, below are the settings I used. Note that the location of func.exe might differ based on the installation type you used.

botasweb

With everything configured, running the project should now start your bot in the func.exe runtime and hook up VS at the same time for local debugging! Awesome!

funcrunning

 

Sample @ github

If you’re struggling, I took the liberty of adjusting the sample project from the blog post and making a bot specific one from it. You can use it to see how it has been set-up. Be aware that my sample is based on the LUIS bot service, so it does require a settings file with your specific LUIS keys to actually work. Should you have any questions or remarks; feel free to leave them below!

Check out the code: https://github.com/jsiegmund/BotAsWebProject

[Azure] Cortana skills + LUIS pre-built domains

A while ago I was planning on doing a few posts on bots, but I never really got to that it seems. I did get into the bot building business though, so here’s one about a little bit more advanced use of bots: language understanding. And although I might write “more advanced”, I don’t really mean it. That’s because LUIS makes things so much easier. LUIS? Yeah, that’s short for Language Understanding Intelligence Service. And it’s awesome. Read More