Amazon Lex V2 console and APIs make it simpler to develop, release, and manage bots so that you can accelerate structure virtual agents, conversational IVR systems, self-service chatbots, or informational bots. You want to evaluate the bots performance over time to gather insights that can assist you adjust the bot to your clients needs. A much deeper understanding of essential metrics such as trending subjects, top utterances, missed utterances, conversation flow patterns, and client belief help you enhance your bot to much better engage with customers and improve their general satisfaction.
In this post, we look at deploying an analytics control panel option for your Amazon Lex bot. The solution uses your Amazon Lex bot discussion logs to instantly create metrics and visualizations. It develops an Amazon CloudWatch control panel where you can track your chatbot efficiency, patterns, and engagement insights.
The Amazon Lex V2 Analytics Dashboard Solution assists you keep track of and picture the performance and operational metrics of your Amazon Lex chatbot. You can use it to constantly improve the experience and examine of end users communicating with your chatbot.
The solution consists of the following functions:
A common view of valuable chatbot insights, such as:
If necessary, Amazon Lex updates your service function with authorizations to access the log group.
The following screenshot shows the resulting conversation log configuration on the Amazon Lex console.
Release the solution.
You can easily install this solution in your AWS accounts by introducing it from the AWS Serverless Application Repository. As a minimum, you offer your bot ID, bot location ID, and the discussion log group name when you deploy the control panel. To deploy the option, complete the following actions:.
Choose View stack occasions to go to the AWS CloudFormation console to see the release details.
On the CloudWatch console, in the navigation pane, pick Log groups.
Select Actions, then pick Create log group.
Get in a name for the log group, then select Create log group.
User and session activity (sentiment analysis, top-N sessions, text/speech technique).
Discussion statistics and aggregations (typical session period, messages per session, session heatmaps).
Discussion circulation, trends, and history (intent course chart, intent per hour heatmaps).
Utterance history and performance (missed utterances, top-N utterances).
Slot and session attributes most often utilized values.
LexConversationLogGroupName– The name of an existing CloudWatch log group consisting of the Amazon Lex discussion logs. The bot ID and area must be configured to utilize this log group for its conversation logs.
If you simply want to test drive the control panel, this service can release a totally functional sample bot. If you want to release the dashboard with the sample bot rather of using an existing bot, set the ShouldDeploySampleBots criterion to real.
The stack might take around 5 minutes to produce. Wait until the stack status is CREATE_COMPLETE.
After you set the wanted worths in the Application settings area, scroll down to the bottom of the page and select I acknowledge that this app develops custom-made IAM roles, resource policies and deploys nested applications.
Choose Deploy to develop the control panel.
You can utilize the control panel to keep track of session period, messages per session, and top session factors.
You can track discussions with a widget noting the leading messages (utterances) sent to your bot and a table including a history of messages. You can expand each message in the Message History section to take a look at conversation state details when the message was sent.
You can envision utilization with heatmap widgets that aggregate sessions and intents by day or time. You can hover your guideline over blocks to see the aggregation worths.
Youre redirected to the application summary page (it may take a minute).
Furthermore, the control panel contains metrics for missed utterances (expressions that didnt match the configured intents or slot values). You can broaden entries in the Missed Utterance History widget to try to find details of the state of the bot at the point of the missed utterance so that you can fine-tune your bot setup. For example, you can look at the session attributes, context, and session ID of a missed utterance to much better comprehend the associated application state.
Utilize the option.
The dashboard supplies a single pane of glass that permits you to keep track of the efficiency of your Amazon Lex bot. The solution is built utilizing CloudWatch features that are meant for tracking and functional management purposes.
The control panel displays widgets showing bot activity over a selectable time range. You can utilize the widgets to imagine trends, confirm that your bot is performing as expected, and optimize your bot configuration. For general details about using CloudWatch control panels, see Using Amazon CloudWatch control panels.
The dashboard contains widgets with metrics about end user interactions with your bot covering data of sessions, messages, sentiment, and intents. These statistics are useful to keep track of activity and determine engagement patterns. If you desire to see belief metrics, your bot should have belief analysis allowed.
As a minimum, you provide your bot ID, bot place ID, and the discussion log group name when you release the dashboard. BotLocaleId– The bot location ID associated to the bot ID with this dashboard, which defaults to en_US. To get the locales set up for your bot, select View languages on the very same page where you discovered the bot ID.Each control panel produces metrics for a particular place ID of a Lex bot. If you want to deploy the dashboard with the sample bot rather of utilizing an existing bot, set the ShouldDeploySampleBots criterion to real. You can use the widgets to imagine trends, validate that your bot is carrying out as expected, and optimize your bot setup.
The source code of this option is available in the GitHub repository.
There are a number of blog posts for Amazon Lex that likewise explore tracking and analytics dashboards:.
Scroll down to the Application Settings area and enter the specifications to point the dashboard to your existing bot:.
Youre rerouted to the create application page on the Lambda console (this is a Serverless service!).
The following diagram illustrates the service architecture.
On the Amazon Lex V2 console, from the list, select your bot.
On the left menu, choose Aliases.
In the list of aliases, select the alias for which you want to configure conversation logs.
In the Conversation logs section, select Manage discussion logs.
For text logs, select Enable.
Get in the CloudWatch log group name that you developed.
Choose Save to start logging discussions.
Now we can make it possible for the conversation logs.
Rich visualizations and widgets such as metrics charts, top-N lists, heatmaps, and utterance management.
Serverless architecture using pay-per-use managed services that scale transparently.
CloudWatch metrics that you can utilize to set up CloudWatch alarms.
You may require to wait a few minutes for data to be shown in the control panel.
You can look for a direct link to your dashboard on the Outputs tab of the stack (the DashboardConsoleLink output variable) when the stack production is complete.
You can add missed out on utterances to intents of the Draft version of your bot with the Add Missed Utterances widget. For more details about bot versions, see Creating variations. When you deploy the solution, this widget is additionally added to the control panel if you set the ShouldAddWriteWidgets parameter to real.
This area explains the CloudWatch features used to produce the control panel widgets.
The dashboard includes custom CloudWatch metrics that are created utilizing metric filters that extract data from the bot conversation logs. These custom-made metrics track your bot activity, consisting of variety of messages and missed out on utterances.
The metrics are gathered under a customized namespace based on the bot ID and area ID. The namespace is called Lex/Activity/<< BotID>>/<< LocaleID> > where << BotId> > and << LocaleId> > are the bot and area IDs that you passed when producing the stack. To see these metrics on the CloudWatch console, browse to the Metrics section and try to find the namespace under Custom Namespaces.
In addition, the metrics are classified using measurements based upon bot qualities, such as bot alias, bot variation, and intent names. These dimensions are dynamically extracted from discussion logs so they automatically produce metrics subcategories as your bot configuration changes over time.
You can utilize numerous CloudWatch abilities with these custom-made metrics, including alarms and anomaly detection.
Comparable to the customized metrics, the solution develops CloudWatch Contributor Insights rules to track the special contributors of highly variable data such as utterances and session IDs. The widgets in the control panel utilizing Contributor Insights rules include Top 10 Messages and Top 10 Sessions.
The Contributor Insights rules are utilized to create top-N metrics and dynamically develop aggregation metrics from this highly variable data. You can utilize these metrics to determine outliers in the number of messages sent out in a session and see which utterances are the most typically used. You can download the top-N items from these widgets as a CSV file by picking the widget menu and selecting Export factors.
The dashboard uses the CloudWatch Logs Insights function to question discussion logs. Numerous widgets in the control panel including the Missed Utterance History and the Message History usage CloudWatch Logs Insights queries to create the tables.
Additionally, the Logs Insights widgets have a link that can take you to the CloudWatch Logs Insights console to edit and run the inquiry utilized to generate the results. You can access this link by selecting the widget menu and selecting View in CloudWatch Logs Insights.
The Lambda functions query your bot conversation logs utilizing the CloudWatch Logs Insights API. The functions then utilize code to aggregate the information and output the HTML that is displayed in the dashboard. It gets bot configuration information (such as utterances and intents) utilizing the Amazon Lex V2 APIs or dynamically extracts it from the query outcomes (such as slots and session characteristics). The control panel uses custom widgets for the following widgets: heatmaps, conversation course, include utterances management form, and top-N slot/session characteristic tables.
The Amazon Lex V2 Analytics Dashboard Solution is based upon CloudWatch features. See Amazon CloudWatch rates for cost details.
To tidy up your resources, you can erase the CloudFormation stack. This completely removes the control panel and metrics from your account. For additional information, see Deleting a stack on the AWS CloudFormation console.
In this post, we showed you a solution that provides insights on how your users engage with your Amazon Lex chatbot. The solution uses CloudWatch features to create metrics and visualizations that you can utilize to enhance the user experience of your bot users.
The Amazon Lex V2 Analytics Dashboard Solution is provided as open source– utilize it as a beginning point for your own solution, and assist us make it much better by contributing back fixes and features. For professional help, AWS Professional Services and other AWS Partners are here to assist.
We d enjoy to speak with you. Let us understand what you believe in the comments area, or use the concerns online forum in the GitHub repository.
About the Author.
Oliver Atoa is a Principal Solutions Architect in the AWS Language AI Services group.
BotId– The ID of an existing Amazon Lex V2 bot that is going to be utilized with this control panel. To get the ID of your bot, find your bot on the Amazon Lex console and look for the ID in the Bot details section.
This post was inspired by the concepts in those previous posts, however the present service has actually been upgraded to work with Amazon Lex bots developed from the V2 APIs. It also includes new capabilities such as CloudWatch custom-made widgets.
Enable conversation logs.
Prior to you deploy the solution for your existing Amazon Lex bot (created using the V2 APIs), you ought to enable discussion logs. You can skip this action if your bot currently has discussion logs allowed.
We likewise provide the option to release the solution with an accompanying bot that has conversation logs allowed and a set up Lambda function to create conversation logs. If you simply want to evaluate drive this service without utilizing an existing bot or configuring conversation logs yourself, this is an alternative.
We initially produce a log group.
The service uses the following AWS services and features:.
Choose Launch Stack:.
Pick the Deployments tab to enjoy the release status.
You can look at a chart consisting of discussion courses aggregated by sessions. The conversation course chart is beneficial to picture the most typical courses taken by your end users and to reveal unanticipated circulations.
The control panel reveals tables that aggregate the leading slots and session qualities values. The session attributes and slots are dynamically extracted from the conversation logs. These widgets can be set up to leave out specific session qualities and slots by modifying the criteria of the widget. These tables work to identify the leading worths provided to the bot in slots (information inputs) and to track the top customized application information kept in session qualities.
BotLocaleId– The bot locale ID associated to the bot ID with this dashboard, which defaults to en_US. To get the areas configured for your bot, select View languages on the exact same page where you found the bot ID.Each dashboard produces metrics for a specific locale ID of a Lex bot. For more information on supported languages, see Supported locales and languages.