A quick test of dozens of hypotheses: how we break out of routine and arrange a discussion in another city

In each of our six development teams, the backlog is scheduled for about two years ahead, taking into account the almost inevitable refactoring, fixes, features and Wishlist of product engineers. Inside, everything can go according to priorities, and some large blocks either become more important, then they are removed, then something new is added there. But there is always an understanding of where to dig in the coming months.



In addition to a bunch of advantages, such a system has two obvious disadvantages:

1. This is trivial boring, and boredom is not what motivates us to write code.
2. Many hypotheses are accumulating that, according to the usual process, fall somewhere at the end of the backlog, but each of which can give a quick result. Or maybe not. Some of them are interesting.

In normal mode, it is difficult to analyze these hypotheses, because you need the interaction of a product scientist, a business person (usually a line manager) and a couple of people from other development teams. That is, when you have two free hours, you still can’t do it. And since we often make money by trampling the path in business to new interfaces and new features, hypothesis testing is life.



First, we set aside time inside the office and did the overall workflow. It turned out that if you check as quickly as possible, you get average solutions. To improve quality, you need to break out of the general routine.



Therefore, we have already traveled to a foreign city three times and worked there.

Why is this needed?

If you have already read posts about how we accumulate technical debt and what we do with it, and about what the developer needs to know about business , you know that from time to time we have dozens of hypotheses about what to do. About half are from developers, partly from stories from other departments and shifting experience to oneself, partly from a product scientist, founder, or simply because of the phase of the moon.



Next, we determine the list of people who are needed to solve most of these tasks. As a rule, this is a specific development team (directions of the air, railways, tours, trains, adventures or analytics), an infrastructure engineer, a couple of people from a business (for a general picture and assessment of financial consistency), an analyst for transferring to and fro, people from other teams .



The basic process looked like this: take a marketer, developers, designer, analysts and leader. Right at the office, once a week we arrange a discussion of the sprint on hypotheses. One day stands out when the work is carried out only on hypotheses. If the solution of one problem leaves in six hours, then it is interrupted and leaves this process. The task is to launch the oblique beta in six hours. Ten hypotheses per week are tested for all teams.



This works well, but the limitations you saw above.



A full-fledged development takes what the project manager is happy with based on the beta results.



We consulted with the project management guru, and several different people said at once that special forces should be set up inside the company, that is, those who are specifically involved in unscrewing quick hypotheses. Somewhere this is called a development group, somewhere else. The point is a permanent hackathon for one team.



It sounds logical, but not quickly implemented: it is necessary to gather experts in six different areas of the business and deprive the main teams of the most interesting, because all the raisins from the rolls are picked by this "special forces".



"Special Forces" is needed because if not 100% of the developer’s time is allocated to solve the problem, it turns out worse. Judging by experience. But we could not do that.



We took TRIZ and suggested that we need to focus part of the time on hypotheses, part time on the main work “in the long run”. What prevents us from doing this, as we did in the office? Context, constant distractions and regulations, constant employment of team members and long feedback.



This is how people came up with hackathons. They remove all office restrictions and give time to focus. Only a hackathon is a free voluntary story, and it is usually about training. Developers spend their Saturday because they can learn something new, and not better do their job.



Therefore, we decided to conduct an experiment and go to Istanbul with a team of 14 people.

Istanbul experiment

Why Istanbul? We needed a city that met the food requirements:

1. Get a flight quickly without frequent delays (the other side of the planet does not fit, islands with unpredictable weather do not fit).
2. Get relatively inexpensive (Iceland is usually not suitable).
3. The city is more attractive than Moscow at this time of the year (not everyone likes to get out of the office for the sake of just traveling, Omsk is not suitable, but the inhabitants of this city will forgive me).
4. There are many interesting things for which you do not have to travel far (Norway is not suitable).
5. The team unanimously wants to go to this city.

We made a list of suitable cities and discussed with everyone. It was important that everything was checked out at once, and this was not a terrible duty.



We decided that we would have a big meeting in Istanbul in exchange for two days off (paid), but we buy tickets ourselves. This logic suited everyone, because this is a chance to dock these two days off with weekends and get a mini-vacation in the middle of the year.



Well, in the end, we are still people passionate about travel.



On the spot they rented a big house near the center.



Who did? One of the developers spent his personal time organizing all of this. I’m not sure if this was because he wanted to study the process of business travel (we just promoted it), or simply because he wanted to help everyone.



For a week they warned the kitchen that we need snacks on the road, but the load on food in the coming days will decrease.



We worked on Friday morning, as usual, at 12:30 we went to the airport, at about 15 o’clock - departure, at 18 o’clock we were there.







They came to the house, there was already wi-fi deployed, I had printed materials with me. All with corporate laptops. We ate, sat and discussed the main things. In fact, it was a debate on what and how to do in the product. That is, we decided what should get into the backlog. I wanted to discuss “quick” hypotheses, and fate, and the priorities of long-term tasks.



The next day we came to this format: on Thursday a list of problematic questions appeared (in addition to those that were already known), so we talked about them almost all Friday. Two sides were found: one was for the hypothesis, the other was against. Then they arranged a duel, which everyone else judged. That is, almost like a trial of hypotheses: the prosecutor pulls for what you don’t need to do, and the lawyer pulls for success and joy. True, then they did not think of changing the prosecutor and the lawyer, and this is a standard procedure in such debates.



The working schedule was this: we choose the worst time for walks (in Istanbul this is the middle of the day), we put the meeting there. Morning and evening are free, but we have lunch together, this is an opportunity to communicate. On that trip, some still finished small tasks, that is, they could not completely turn off the usual process. On the other hand, I would not say that it took a considerable time.

Example of a run-in hypothesis

The buses do not have a legally approved electronic ticket. This saddens us incredibly, because passengers must either print the form at home or print on a printer at the bus station (which sometimes becomes a problem, and sometimes it is banal for a fee). Russian Railways has long accepted electronic registration almost everywhere, airplanes print to you at the airport without questions at the terminals or at the front desk (and in some places you do not need to print). And in buses in some places still the 70s of the USSR.



In practice, there are progressive stations that just show a ticket from the phone. They still have this data in the statement on their part, and you just need to look there and at the person’s document. But there are conservative stations and those who are just “Baba Yaga against.” And all stations have their own forms of such forms.



From the station’s point of view, an electronic ticket is a development. The safety of the station increases, it is more convenient for the controller and the driver, operations are accelerated, paper is saved, young people do not ask questions.



Anyway, on a bus, one or two passengers forget the tickets, and in any case, they are restored from the station data. But in some places they find fault very much. Practice has shown that if a passenger begins to scandalously, they let him in. If he leaves quietly (most often pensioners) - we have to understand the situation already for us.



The first thing we did was to allocate conservative stations and when buying tickets we make a separate notification with them to the passenger that it is necessary to print a ticket: they won’t be allowed without it.



Then we decided to make a “white list” of such bus stations where electronic registration works. Threefold check: passenger recall, call of our secret customer, direct question to the station management.



If the legislation lags behind reality in terms of allowing electronic tickets, but there is a workaround through ticket recovery according to the station, then why not automate and make your own quick and convenient crutch?



In general, we found such stations and marked tags on the site.







Verification is repeated from time to time.



In the process, it turned out that there were stations that themselves came to such a scheme, but did not really tell the passengers about it. Integrators to this also go with joy.



Then we gave small bonuses in the system to those stations that are marked with such a mark that there is an economic incentive to do so.



As a result, it turned out that we quickly combined (well, actually not us, but they themselves combined, in particular, with our tool) quite a few stations and carriers that already do electronic registration themselves.



That is, you could not just sit and wait until everything appears legislatively, but figure out how to do it programmatically. And that’s it. We needed a bunch of a developer, manager, a person in communication with partners and a designer.

What is the result?

Losses of the first experience:

• Minus tickets and accommodation.

Benefits:

• Almost like a vacation in the middle of the year.
• For the next six months, they decided all the questions at once.
• They were able to communicate precisely with the team. For some reason, the office does not work out that way.
• Difficultly measurable things at the level of sensations: relations in the team have changed.
• We looked at our own product from the outside: after all, we used our tools (and other teams) all the time, came up with several features “on the fly”.
• Just drove on a trip, which is logical for a company that is about travel.
• The "senior" comrades taught the "Junees" to think rationally, not only in development, but also in planning for the day. This is very insignificant, but the transfer of experience is felt.
• They were able to pull remote developers into communication, which was usually not enough.

Nezhdanchiki:

• We learned that the two girls had catastrophic problems with orientation to the locality: they were lost, we found them, never let them go. This nearly disrupted the working session.
• The developer who took on the organization showed a manifestation of situational leadership. Not quite expected, eychar and leader were surprised.
• We found out that our colleague Misha takes cool pictures. He took a camera, took pictures of everyone, then presented paper prints. For memory.

It is very difficult to synchronize on trips, and this has not yet become a process. But I think it will, because the benefits are obvious. Now we use both approaches: we allocate the time of teams in the office to analyze hypotheses and occasionally leave. Departures are needed more to determine vision and different tasks, and “do not disturb” in the office to break in hypotheses in the personal hackathon mode.



Seven hypotheses were selected and tested in the first iteration, of which three showed results. For example, in the direction of buses.

At the stage of data entry, passengers began to show a plate with the inscription "Last ticket purchase for this flight N minutes ago." On the mobile version A / B showed + N% to sales, on the desktop there are no significant results. We expanded the search form on the mobile version of the pages with the schedule - as a result, we got + NN% to sales. We receive data from customers in order to be able to return them. On empty issues, they began to collect user emails to send buses, if they appear in the direction, there are hundreds of them per week. Based on user preferences, we collect offers in the newsletter. The results. Hit: 91–93%. Sales from those who opened the letter: + NN, 3% (significant). But! People ride buses in the same directions, which means that the predictions are the same. So far, it turns out that the mailings will always be the same. We will think how to diversify.

At that time, there were typical tasks in the backlog, for example, such a routine:

• Run two integrations with bus stations.
• Update three current integrations.
• Integrate Lithuanian bus carriers.
• Make connectors for your account 16 carriers.

In general, it works, but we would love to hear your experience of working “in isolation” from the office and traveling somewhere if you had them.