In the remote channels of Alfa-Bank, we also develop products. But in our teams there are no other analysts, other than system analysts, whose duties include collecting and analyzing requirements, as well as designing and documenting solutions. An attentive reader will notice a slight contradiction here. So what does it mean that system analysts are also actively involved in product analytics?
Under the cut - a small study, which was designed to assess how strongly system analysts are involved in the process of working with product metrics.
Polls
As in other cities and towns of our country, the owner of the product deals with food issues, so everything related to metrics is his diocese. And therefore, to dive into the context, we conducted a survey among the product owners of the new Internet banking for individuals and legal entities. We had three main blocks of questions.
- Who and how in your team determines product metrics?
- Where do you get metric data from? What sources do you use?
- Which of the team members (roles) is involved in the processes of determining metrics, collecting and analyzing data to monitor their achievement? What is their participation manifested in?
It turned out that product owners distinguish three types of metrics.
Business metrics. As the name implies, they show how much the product made money or how much money the bank saved. They are determined by the owner of the product or set immediately at the strategy level. Data on such metrics is downloaded from several sources, read - with the involvement of the development team. But the analysis of this data is on the shoulders of the product owner and business experts.
Usage metrics. They help to understand how end users actively work with product features. Is it convenient for them to move from section to section, view prompts, press buttons and more. These metrics are determined by the product owner in close collaboration with the development team. And in order to monitor this data in a convenient format, the development team sets up dashboards.
Health metrics. They help to keep a finger on the pulse of the product itself, to notice whether errors, memory leaks and other not very pleasant things appear. For the most part, they are similar to the previous type of metrics, it's just that the product owner is not so actively involved in the process of determining and analyzing them.
After that, we already interviewed system analysts for their involvement in working with product metrics. And yes, it was the guys from those teams that were interviewed with whose product owners they spoke at the first step. There were 4 questions with pre-prepared answer options.
# 1 How often do you face data collection and analysis tasks to control product metrics?
More than half of the guys (58%) answered that they do this on an irregular basis, that is, you sit and do what system analysts usually do, then the owner of the product comes to you and asks you to upload the data upon his request. 17% of respondents said that this cup passed them at all, so we did not begin to ask them the following questions.
Since the product owners reported that the development team is downloading data from different sources for analysis, the second question was:
# 2 What sources do you use to collect and analyze data to control product metrics?
The most popular source was the Automated Banking System, this is with us - Equation (35%). Silver took the Alpha Metric base with 26% (we at one time made our own analogue of Yandex Metric for similar pieces).
(ALBO = Alpha Business Online)
The third question concerned tools.
# 3 What tools do you use to analyze data to control product metrics?
Top predictably occupied SQL and Excel (53% and 40% respectively). 7% preferred Python, but this is more likely an exception to the rule.
# 4 What data monitoring tools do you use?
It tastes and color - there are three main tools at once.
- Superset (40%) - used to build dashboards based on data from the Alpha Metric database;
- Kibana (33%) - used to create dashboards based on data from application logs, as well as data from local product databases;
- Grafana (20%) - often used to create dashboards for monitoring health metrics.
So what is the conclusion?
A system analyst is faced with the task of collecting and analyzing data to control product metrics. However, they are irregular in nature, therefore, most likely, they do not distract the system analyst from the main tasks. And the analyst uses different data sources to control product metrics. It has direct access to a part of sources, from other data it is necessary to receive according to the application. To simplify the life of the analyst, it would not be out of place to get away from applications by giving him access to the necessary sources.
The main data analysis tools for monitoring product metrics are SQL and Excel. System analyst does not need other specialized tools for data analysis, he is involved in the data monitoring process to control product metrics and uses different tools for this. It might be useful to move to a single tool for monitoring data.
Recently, my colleague wrote an article in which she tried to answer the question of what a system analyst at the Bank does. The analyst, like other members of the development team, can really deal with product issues, including work with product metrics.
But such tasks are more supportive in nature, aimed at helping the product owner upload data for analyzing business metrics, analyzing usage metrics and generating ideas for developing product functionality, ensuring stable operation of the application by analyzing and monitoring health metrics.