A year ago, our beloved HR department requested: write a bot chat that will help in adapting newcomers to the company.
We make a reservation that we do not develop our own products, but we provide clients with a full range of development services. The story will be about our internal project, for which the customer is not a third-party company, but our own HR. And the main task with limited availability of people, resources, time to complete the project on time and release the product.
First, we describe the tasks that were to be solved.
Developers are mostly introverted people and do not like to talk, it is much easier to write your question in electronic chat. With a bot, you don’t have to think about who to ask, who to call, where to go and in general, where to look for information, and whether it is relevant.
The second problem is information - there is a lot of it, it is in different sources, it is not always there and needs constant updating and updating.
The company has almost 500 employees, they are located in different offices, time zones, cities of Russia and even abroad, there are usually a lot of questions, so another task is to reduce the burden on HR personnel associated with the most common questions asked by employees.
It was also necessary to automate the processes: the arrival of newcomers to the company, sending messages to managers and mentors of newcomers, sending automatic reminders of courses and tests that a newcomer must pass to successfully adapt.
Based on business requirements, technical requirements were formed.
The bot should work on the basis of Skype (historically it has developed that they use it in the company), therefore, a service on Azur was chosen.
To limit access to it, we began to use the authentication mechanism via Skype.
For recognition of the text used the library ParlAI
An administrative web portal is also needed for setting up, training, debugging, setting up mailings and other tasks.
In the process of working on the project, we encountered a number of problems and difficulties.
For example, there were technical problems - with an account for Azure. Microsoft did not want to activate our subscription due to any technical difficulties inside their service. For almost two months, we could not do anything with this, Microsoft support finally shook hands and sent us to partners who successfully set up everything and gave us an account.
The most difficult stage was the start of the project, when you need to choose what we use, what architecture will be, how and where to store data and how the components and modules of the system will interact with each other.
In our case, in fact, the ordinary problems of starting any project were complicated by staffing as well. The specifics of our business is such that, unlike commercial ones, developers who often do not have sufficient knowledge in the necessary areas often work on internal projects - they just happened to be on the bench waiting for the next big cool commercial project. It is logical that with motivation in this situation, things were also very difficult. Performance drops low, the team is often simple, in the end you have to persuade (motivate) or change a person. When changing a developer, you need to conduct training, transfer knowledge and, again, essentially start a project. Each new developer saw architecture in his own way and scolded the previous ones for the decisions they made and the code of others. The rewriting began from scratch.
This went on for about six months. We just stomped on the spot, refactored the code and wrote nothing new.
Also, on internal projects, as a rule, there is almost no documentation, and it was difficult to understand what needs to be done at each moment of time, and what are the priorities now. It was necessary to create a permanent team, establish processes, carry out planning and evaluation for at least three months. But how to do this when the project is not commercial, which means that you need to invest human hours at a minimum, and at the same time get the result no worse than for an external customer?
We have identified a pool of resources that participated in the development of the project, are familiar with it and want to work on it. Made a schedule of employment of people on projects. We evaluated and coordinated the work, and entered the work in the “holes” between the main projects. After 4 months, we received a working prototype of the application.
Now let's talk in more detail about the functionality of the bot, about architecture and technical solutions.
One of the main requirements of HR was the recognition of text written by the user to correctly answer the question. You can write to him - I want to go on vacation, I want to go on vacation or go on vacation, and he will understand and respond accordingly. Or suddenly the employee’s chair broke and he wants to write - “the chair is broken” or “My chair is cracked” or “The back of the chair fell off”, with proper training, the bot will recognize such requests. The quality of text recognition itself depends on the bot's training, which we will talk about later.
The next requirement and part of the functionality is the bot dialog system. A system was developed in which the bot can conduct a dialogue and understand the context of the current issue. In response to your question, he can ask any clarifying questions and continue the conversation if we trained the bot to do this. Skype supports simple menu items to prompt users about options for continuing dialogs. Also, if we had a dialogue, but suddenly decided to ask a question off-topic, then the bot will also understand this.
The bot makes it possible to send various artifacts to the user, based on his personal data. For example, at his location. Suppose if a person wants to find a toilet, he will be shown a map of the office leading him to the toilet. And the card will be selected depending on which company’s office the employee is in.
One of the most important tasks is to protect the personal information of users. We cannot allow everyone to access the confidential data operated by our bot. The need for authorization for such a bot is an integral part of it. The bot asks the user to go through authorization before he can conduct any dialogue with him. This happens when the employee first contacts the bot. The authorization itself redirects the user to the appropriate page, where the user receives a token, which he then inserts into the Skype message. If authorization is successful, then you can start communication with the bot.
Authorization passes through Skype - portal - authorization service, corporate network and LDAP. Thus, authorization depends on the current user data in the corporate network.
In the process of developing the bot, we realized that we needed a certain system built into the portal’s functionality, which could help HR to quickly debug the bot. We have added such a portal page on which HRs can see the errors recorded by users when working with the bot and solve them with the help of retraining or leave it for developers.
The possibility of training the bot directly on the portal was not laid from the very beginning. During the development process, we realized that bot training is the most common task that HR department employees will perform when working with it, and sending text files to developers for additional bot training is completely unacceptable. It eats up too much time and generates too many errors and problems.
We wrote a UI on the portal for User-frirendly bot training. It allows HR to see the bot's current training, to train it and make adjustments to the current training. Learning is represented by a tree structure in which nodes, that is, branches, are a continuation of the dialogue with the bot. You can create simple questions and answers, or weighty dialogs, it all depends on HR and their needs.
A few words about the architecture of the solution.
The architecture of the solution is modular. It includes services that are responsible for various tasks, namely:
• Skype bot service on Azure - accepts and processes user requests. This is a fairly simple service that first accepts the request and performs its initial processing.
• Admin portal - a service that provides a web interface for configuring the portal and for the bot itself. The bot always refers first to the portal, and the portal already decides what to do with the request next.
• Authorization service - provides authentication mechanisms for the bot and for the admin portal. Authorization takes place using the Oauth2 protocol. With a positive authorization, the service authorizes in the corporate network according to the valid user data, so that the system can control errors associated with the data desync.
• AI A text recognition module written in python that uses the ParlAI framework for text recognition itself. This is a neural network, at least in the current implementation. We use the tfDiff algorithm to understand the issues. The module provides an API for communication and training.
In conclusion, I want to say that this is our first experience in creating a chat bot, and we tried to make the system as simple as possible, but at the same time functional, with minimal labor costs. I think we have a very interesting product. With its system of training, error logging, notification mailings, it can also be integrated with any other messenger.