Alice Award: 5 insights from the winner

October 10, 2019 is Alice's second birthday. Popularizing the technology of voice assistants, Yandex continues to distribute money monthly to developers of voice applications in the Alice Award contest.



In the pursuit of the prize, developers can come in handy the comments of the author of “Associations for a while”, who received the 1st place and 100,000 ₽ in the category “Games and Entertainment” in the September competition.



The rules by which Yandex awards the Alice Prize are known only to Yandex, so it’s hard to say thanks to or contrary to which of the facts and observations described below I managed to be among the winners.



It should also be borne in mind that Yandex is constantly experimenting with the algorithms of the skills catalog, their ranking and recommendations. Nevertheless, I hope this information will help the creators of skills to make them better, more popular and worthy of prizes.

Insight # 1. Be simpler and people will reach for you

In “ Associations for a while ”, as you might guess, you need to quickly guess the words by association. We came up with hundreds of tasks and began to offer them to players at random.



Very soon, from reviews by friends & family (friends and relatives), it became clear that some associations are more complex than others and cause difficulties. Then we ranked the tasks on a scale from 1 (“the child will handle”) to 3 (“have to think”), first offering the simplest of them and requiring fewer correct answers to move on to the next round.



After adding a chat for players in Telegram (in which no more than 20 participants were added in 2 months), we began to receive feedback about difficult tasks directly from Alice's users. Judging by their reports, a significant part of the players were children. When asked to evaluate the complexity of the tasks from 1 to 10, we received the answer “13”. This forced us to reconsider our scale of complexity and leave only almost obvious, in our opinion, associations in the list of simple ones.



Participating in the “Aloud Games” hackathon, we again received feedback about the complexity of the game for new users and the idea not only to start with simple tasks, but also to show the right answer for the first few questions, so that the player only had to click on it and, thus , better understand the meaning of what is happening.



As a result, simplification in the form of hints, sorting tasks by complexity and additional time for reflection should positively affect retention (“returnability”), one of the main, according to Yandex employees, indicators of the demand for the skill.

Insight # 2. Rating: ask, and it will be given to you

Retention makes sense if someone already comes into your skill. How to attract into the skill users who are not in the concept of friends & family? Yandex employees compile skill sets and place them on the main page of Alice’s voice application catalog.



At conferences, Yandex representatives drew attention to such an indicator as the number of ratings that users can leave on the skill page in the catalog. It's no secret that you can put your own skill ratings. Moreover, from different Yandex accounts, for the establishment of which it is not necessary to confirm mail or phone, it is enough to come up with a username and password. This path does not seem fair, but in the absence of other users the skill seems to be effective. A couple of dozen assessments of a new skill - and the skill can fall into the “New” collection and get several hundred (in the case of “Time Associations” - 500) users for each day of stay in the collection.



Once the initial flow of users into the skill is initiated, you can count on the ratings of these same users. First, I added a link to the skill on his page in the directory, placing it in a place of dialogue that is pleasant for the player (setting a record). It did not give the expected effect.



Analysis of approaches to collecting marks in skills with thousands of marks showed that modest reference is not enough. The chances of getting a rating can increase if, firstly, you explicitly ask the user to give a rating, appealing to his emotions, and secondly, do it at a time when the user was already able to rate the game (for example, he set a fairly large record) and, thirdly, it is better if at the time of the request the user does not have other hints other than the transition to rating.





By the way, do not ask to rate you developers of other skills. Attempts to post a link to a skill in a community chat usually result in the addition of “units” and a downgrade.

Insight # 3. Hit the Top - Pay

If you manage to get positive marks, you’ll be likely to get into the “Popular” collection, top positions in the skills rating and Alice’s recommendations (her answers to the questions “What can you play?”, For example).



This can attract one or two thousand users daily to the skill, i.e. about 30,000 users per month. You will have to pay for the maintenance of this traffic, including the storage of user data. For example, in the Aimylogic tool that I used, the free tariff is limited to 20,000 unique users per month for all your skills. To provide a load of 30,000 users on Aimylogic, you will have to switch to a paid tariff (59,000 ₽ / month) or to another account.



It should be borne in mind that, for the example of “Associations for a Time”, two weeks of work for the accumulation of Yandex data and the award of the prize is not enough. It is worth counting on 1-2 months of maintaining the skill in working order.

Insight # 4. Not all speakers are equally fast.

In “Time Associations”, response speed is essential. We noticed that the Yandex.Station players managed to answer 9-10 questions in 1 minute, and only 7 in the Irbis column. This is due to delays between the moments when Alice finishes talking and starts listening. On different devices, the delays are different, which significantly changes the UX in games for a while.



In the end, I balanced the number of answers required to advance to the next round, taking into account the slowest device. At the same time, smartphone users have an advantage in the form of the ability to prematurely interrupt Alice's speech by reading the text of the assignment.

Insight # 5. Do and redo

The first time I applied for a prize in July 2019. Unsuccessfully. After that, the skill underwent a series of changes in the reviews of friends & family, the feedback of real users and the comments of experts at the August hackathon “Games out loud”.



During the hackathon, where “Associations for a Time” took second place , experts from Yandex recommended thinking about a game for several players, as well as a simplified mode without taking into account time. At the hackathon we did not manage to make the corresponding changes. However, in September, the game was redesigned in accordance with these proposals.



In September, the application for the award was re-submitted. In the commentary on the application, I drew the attention of the moderators to new features - and the result in the form of 1st place was not long in coming. The presence of the PvP mode in the game was noted in a post with the results of the competition .

Conclusion

For the year of the existence of the Alice Prize, Yandex has given developers more than five million rubles of skills. The rules for awarding prizes are controversial, but the growing competition in the Games and Entertainment category and the information on hundreds of thousands of users of the most successful games in Alice demonstrate that this competition is achieving its goals - both the quantity and quality of skills are gradually growing.



I offer readers a small survey about your attitude to the Alice Award.