Refactor in parallel with development: our experience and two checklists

For many teams, refactoring is a pain. Because if you do refactoring, you don’t develop the main product, and if you don’t do it, the technical debt of the project grows. At some point, the teams come to the thought: “Let's distinguish between refactoring and development and allocate, for example, 20% of our man-hours, and the rest of the time we will continue to develop!” This thought is not bad, the only thing is that every 100 development hours you will never get 20 pure hours of refactoring. Because “development” is not only work with code.



If we are talking about mature teams, rather than mini-teams for 3-5 people squeezed into a material point, then “development” includes a whole host of different activities of the team besides writing code. In the "development", you can write down meetings so unloved by many, work with documentation, maintaining reports in task managers, and so on. All this eats up approximately 30% of our watches allocated for development. And somehow, unnoticed, it turns out that instead of the picture “80 hours of code, 20 hours of refactoring”, we get an unsightly figure of ~ 13 hours for, directly, the refactoring itself, because everything else was absorbed in other activities.



Below we will tell you how to combine development with refactoring so that the technical debt of the project does not grow like a snowball, but also talk about the correct distribution of time and prioritization.



Why does refactoring cause so many problems? First of all, due to the fact that teams cannot adequately evaluate their own resources and combine this process with development. It seems to people that if their working day is 8 hours a day (40 per week), then all 40 hours they are supposedly engaged in development. This is not true.

The whole development process can be divided into two parts: these are side activities and everything that is directly related to the code.





Here's what the time distribution of one of our development teams looks like.



The reader may have a reasonable question: “how did you build this diagram?” And now we will try to give an answer. The data was not taken from the ceiling, but based on our internal statistics. Our developers keep track of their activities at Jira. Each such personal worklog consists of four sections: code review, technical discussions, specific tickets, and “everything else”. Thanks to this fairly simple and transparent classification, we built an analytics of how much time in which teams we spend on each aspect of development. If a third of all working time falls to meetings, stand-ups and reviews - everything is within the normal range. If more than 33% is spent on these activities, then the team has problems and needs to be addressed.



It seems that there is no catch here and everything is logical, but how do we fit refactoring into this story? Declare a "month of refactoring" and score on development? However, any commercial product has its own schedule, which is extremely undesirable to get out of. Still refactoring is like a quagmire: if you started to deal with it, then it's hard to stop, you are drawn to this bottom. Refactoring shifts the focus of the team’s attention to itself, and we get some monstrous bias towards “restoring order” in the already written code instead of moving the project to a brighter future. So how to distribute time?



The following slide gives some answers:





Keep time logs, it's wildly useful



Meetings and reviews remain untouched, because we believe that these activities are optimized as much as possible and kept to an adequate minimum (those who worked in teams where meetings and code reviews took 70% of the time will confirm our words). So, we take the time to refactor the code and bug fixes from the development. And here we again apply the “one and two-thirds” approach and divide the man-hours we received into “refactoring” and “bugs”, clearly separating these concepts. Such a model is viable and allows you to find the time to clean up the project, at least without increasing technical debt. If we bite off too many “development” hours, the project will stall.

We approach the refactoring process correctly

Suppose you decide to refactor instead of rewriting a project from scratch. But here you can’t grab the first one and refactor like in the army, "from here to lunch." Since our resources are limited and our goal is to stop the growth of technical debt (and, ideally, to reduce its size), we must approach this task sensibly.



The best solution seems to allow the team leader or another development manager to determine the scope of work, assign tasks and start refactoring. But this option has a serious flaw. Timlid is the conductor of our team, but the problems of local musicians are not always obvious to the conductor. If you are a supporter of a rigid vertical and a model in which one person will decide how refactoring will take place, then you voluntarily sit in the burning KAMAZ, which with broken brakes rushes from the mountain into the abyss.



Our experience shows that a robust version of the development of events is a collective definition of certain areas of refactoring, that is, developers should take part in compiling a list of future work. Only the immediate author of the code can honestly admit whether this section was adequately closed or because of a lack of time there were piled up various crutches. In addition, it is the developers who are at the cutting edge of technology and are able to sensibly assess which elements of a project need to be refactored and which ones should not be touched.



For ourselves, we came up with this approach: each of the developers remembers the delicate places on the project and writes a card, what needs to be done to make it better. On these cards then tasks will be determined. True, there is a nuance: after the first received card in the style of "doing well" without any context, we began to demand this very context in a compressed form.





The same cards



But defining a task is not enough to begin its implementation. Therefore, for each such card, a checklist should be compiled that answers a number of simple questions like “Do we need to do something in parallel?” Or “Are there any factors that will prevent us from completing this task?” After filling out such a checklist, the developer gets a specific list of all possible problems and blockers that need to be solved at the stage of preparatory work.



Another mandatory step is to determine who is best versed in which areas of the project. This will optimize the refactoring process and provide team members with tasks in those areas where they can best prove themselves. True, once we were faced with a situation that one and a half people normally understood one aspect of the project, but even this problem can be solved. To eliminate the information vacuum, developers who still “rummage” in the area where the rest of the team passes, should share their knowledge. This may be documentation, some kind of mini-presentation, even a video with explanations of the code. It is important to convey information to colleagues and make sure that there are as few dark spots as possible.



Well, the last: the team should clearly answer the question “what is our technical debt?” Grades like “three”, “acceptable”, “five out of ten”, “you can live” no longer work. Earlier, we answered “five out of ten” to a similar question from the service station for one of the projects, and when we converted the size of the technical debt into numbers, we got 650 hours. It was awkward. Only a clear answer to this question will help sensibly assess the scope of upcoming work. This is important because “endless” tasks kill the motivation of team members, as well as enrage business: both developers and management should see some tangible endpoint that the team will strive for.

Prioritization

After we have defined the front of work, we need to combine refactoring with development. Obviously, we cannot stop the sawing of features in different parts of the project or allocate some special time for this event.



This is where the most difficult part begins. The development team should sensibly determine in which moments development is more important and in which refactoring is important, especially when some feature is being sawn right at this point in the code. Perhaps, after refactoring, further development will go easier, and in some cases we can finish our feature and only then refactor. We communicate with the product, argue the decision and agree on what the order will be. For example: “Let's first refactor - not just because we feel like it, but because the total term will be less in the end.”



The most important thing is not to forget that our goal is to develop a product, and not to write the perfect code for the sake of the perfect code or the complete elimination of technical debt. Development should be fully adequate in this regard: if refactoring right now hurts our business development task, then we finish writing a piece of new code, and only then we correct the old one. If circumstances say that it will be more profitable for the project to refactor this section (no matter how monstrous the front of the work looks like), since further work will be simpler and more fun, then we need to refactor.



This approach fits extremely poorly into the vertical management model and sometimes you will encounter situations when you have to defend your decisions in front of the customer. But otherwise nothing.

What do we get as a result

With the described approach, we organically integrate refactoring processes into the current development, reduce the size (or stop growth) of technical debt, and all this without any major casualties.



Yes, for the sake of refactoring, you will have to reduce the time to develop new features, which will increase the time, but the choice is small: you either refactor in parallel with the development, or sooner or later come to such a heap of crutches, problems and bugs that it will be easier to rewrite the project from scratch. So if you value the work of your team, you should still think about refactoring instead of rewriting in the future.



Useful materials:



1. Alexey Kataev's report on refactoring: more examples from practice, as well as an amazing story of manager Gleb, who did not want to refactor until ...







2. Our regulations for discussion of tasks



3. Our checklist code review