Gal Shenar is a professional software engineer who attended an Alexa event where he learned about the potential of Alexa skills. He started out building workout and lifestyle skills because they were simpler and it was easier to provide a great user experience. As those began to succeed and bring in Amazon Developer Rewards, he founded Stoked Skills and began to focus on making great voice-based games for Alexa.
Shenar’s goal is to build the highest-quality voice user experiences for Alexa, and for Stoked Skills to become a major player in skill store.
To determine whether or not users were engaged in the user experience he created, he was diligent about which success metrics he tracked using Dashbot. He points out the differences between analytics for mobile apps and skills, “With voice devices, there are no apps that are saved on a digital screen. You have to remember it in your head, and sometimes it’s hard for users to remember the invocation name. You can increase the length of each session and increase the number of users, but you have to get people to come back.”
Shenar emphasizes that because of these user challenges, retention is the best way to measure if you’ve built something people like and really enjoy. For new skills he releases, he checks Dashbot’s Live Transcripts and the number of users to understand performance and debug any errors. For existing skills, he’s more interested in increasing his Alexa skill directory ranking and moving the needle on reviews so he digs into Dashbot’s Competitive Intelligence (a tool that shows how your Alexa skill compares to other skills in terms of reviews and page ranking).
Improving the UX Improves Alexa Skill Ratings
Shenar first started using Dashbot when he had released Escape the Room, a voice version of the popular “Escape Room,” because he really wanted to improve the skill since he saw a lot of potential in the games category. The skill had a rating of 3.3 out of 5 stars in December 2017. By using Dashbot’s live transcripts and tracking user sessions, Shenar was able to identify a major issue with the skill that was directly affecting his user reviews.
Amazon’s voice recognition service was incorrectly identifying objects in the room that the user was trying to inspect during the game. Shenar had set up custom slots to only accept objects in the room, which meant that Alexa should only be matching those terms. Unfortunately, in many cases, the skill wasn’t able to match those words correctly and was identifying items like “nail file” as “whale file” and “backpack” as “back back”.
In order to fix this, Shenar examined all of the occurrences of incorrectly identified items in Dashbot’s live transcripts and manually matched the items people were trying to interact with. Then, he built a system where he could match a list of those incorrectly identified terms with the real name of the item. Shenar notes that “Discovering the issue through Dashbot and making that fix was one of the major contributors to the review score increasing from 3.3 out of 5 to 4.6 out of 5 in the next few months.” Escape the Room now has nearly 1,000 reviews and an average rating of 4.6.
Another significant success metric Shenar keeps an eye on is the percentage of new versus returning daily users. “Increasing the count of returning users is a goal of mine that helps to track whether or not the game can sustain itself. With the debugging fixes, as well as some other gameplay tweaks, I was able to increase the percentage of returning users per day from around 10% to 25%,” says Shenar.
Shenar plans to continue developing new skills and improving his existing ones. He is especially excited about Amazon’s new monetization feature, in-skill purchases like subscriptions and one-time purchases, and plans to incorporate them into his skills. his most recently published Alexa skills, Escape the Airplane and Word Play, so go check them out!
Dashbot is a conversational analytics platform that enables enterprises and developers to understand user behaviors, optimize response effectiveness, and increase user satisfaction through actionable data and tools.
In addition to traditional analytics like engagement and retention, we provide chatbot specific metrics including NLP response effectiveness, sentiment analysis, conversational analytics, and the full chat session transcripts.
We also have tools to take action on the data, like our live person take over of chat sessions and broadcast messaging for re-engagement.
We support Alexa, Google Home, Facebook Messenger, Slack, Twitter, Kik, SMS, web chat, and any other conversational interface.