Creating an application that can understand natural human language used to be a complex affair, but thanks to Microsoft’s LUIS (Language Understanding Intelligent Services) application the ability to program natural language understanding into any application has never been easier. LUIS’s easy to use interface allows people from all different technical backgrounds to quickly and easily create a custom language solution.
Part of the process of creating a great LUIS application is the frequent refinement of the intents. By adding user created examples to your intents you can create a more robust application. As of right now, there isn’t a great way to visualize this data. This is where Power BI can help us gain powerful insights into how people are interacting with our LUIS application, and facilitate our refinement process.
In this example, we are going to use Power BI to visualize the past month of endpoint utterances and see if there are any insights we can gain. For this example, you will need: a LUIS instance on Azure (there is a free subscription tier), the app Id, subscription key, resource location, and Power BI Desktop. I have uploaded the Food Truck LUIS example from the Microsoft GitHub into my LUIS instance and queried the endpoint 10 times with various food ordering queries.
LUIS has a suite of authoring API’s that we can use to extract data about our LUIS application. The one we will be using for this example is the ‘GET: Download application query logs’. This API call brings back a CSV file containing the past month’s endpoint utterances. In Power BI we can get the information from this API call be using the ‘Web’ data source. Under the advanced menu we will enter in the request URL which is formatted as follows. Be sure to add your location (i.e. westus, eastus2, etc.) and your app Id in the corresponding fields:
Also include the request header ‘Ocm-Apim-Subscription-Key’ field. Use the access keys that you can find on your LUIS instance in the Azure Portal.
The data you pull in should look something like this:
Click on edit the data so we can format this information in an easier to understand fashion. To make the next parts easier we are going to add an index column. In the ‘Response’ column, expand everything out. This will create more rows, but because of our index column each item will keep its correct index. Delete the unnecessary columns. We should now have an index column, the query, the date time is was submitted, the intent, and the intents score.
For this example, we only care about the highest scoring intent so we will group all the columns by the highest score. We will also include a reference to all the other columns for future reference.
We can then expand the table that gets created under ‘All’. This gives us the ‘High Score’ for every row and the actual scores. By including a condition column that is true when the ‘High Score’ and actual score match, we know that record must be the labeled record.
Now in the future we can filter based on the ‘Labeled’ queries to get the highest scoring label. We can clean up the data by deleting unnecessary columns, renaming columns, and changing column types to the correct values. The result should look something like the table below:
We can now start creating visuals. In this example I have created a pie chart with every labeled intent, and the number of each query that got labeled to that intent. By clicking on each section of the pie chart, we can see the types of queries that fall into each. For instance, in the ‘None’ intent we have a query asking for veggie burgers. If the restaurant does sell veggie burgers, we may want to add an example to the ‘OrderBurger’ intent about veggie burgers, so that it can be more accurately categorized in the future. We also, have a request asking where the bathroom is. At the moment, the LUIS application doesn’t have a ‘FindBathroom’ intent, but it may be something we want to add. Using Power BI, we can track and see it this is a frequent request, and if it is worth adding to the application.
By visualizing this information in Power BI, we were able to quickly and easily identifies ways we can improve our LUIS application. The LUIS authoring APIs offer a wide variety of ways to get data about your application, and this is just one example. Creating a great LUIS application takes repeated refinement, and by using Power BI we can greatly bolster our analysis.