Estimation of the project term. Why is it almost always very understated and what to do about it
When calculating the project period, we traditionally evaluate the duration of the intermediate steps, then we sum them up and add a buffer for all sorts of randomness. Then the leadership cuts this term for us in half. As part of this note, the author will be interested in our calculations, because even a project manager with extensive experience often understands that the calculated period is too short and much, sometimes at times, at odds with his personal expert assessment. Yes, he will correct the estimates of the terms of the project and the intermediate steps to his expert assessment and, with true mastery with some processing, will meet the deadline with an accuracy of 15%, but the sediment will remain.
This note explains the reason for the discrepancy between expert and theoretically calculated estimates. It is also considered why the “overstated” expert assessment is usually underestimated if it is not done on the basis of statistics on the implementation of similar projects. In the end, it is revealed how to correctly calculate the project term and explain the situation to interested parties before the start of the project or during the project.
When we evaluate the implementation period of a piece of work, we determine the most probable figure. It can be an expert assessment on one point or on three points, as in PERT. In a complex project, it can be parametric modeling. In all cases, we make the same mistake. The fact is that in all classical methods of estimation it is assumed that we have a normal distribution of the estimated value - according to Gauss. Theorists are very fond of this distribution.
Normal distribution means that a project with a correctly estimated duration of 6 months is equally likely to end in 9 months or 3 months. At equal distances from the distribution center, the probability of completion of the project is equal - this is a feature of the Gauss curve. On the other hand, in practice, few people saw an IT project that completed twice as fast (after 3 months), but projects that are delayed by one and a half times in terms (9 months) we see regularly. In addition, with a normal distribution, half of our projects would end before the estimated time, and half after. But this is not observed in practice either. This suggests that the normal distribution for estimating deadlines is not applicable. That is, we do not have a normal distribution, but some other, with different probability of completion before the most probable date and after.
Consider the lognormal distribution as an example of such a distribution. It will show us the features of this class of distributions. For the lognormal distribution, the median and mathematical expectation are significantly beyond the peak:
On the graph presented, the peak shows the greatest probability of the project completion date. This figure usually includes this figure plus a certain margin. The median indicates a separation point at which half of the projects will be completed before this deadline, and the second half after. Mathematical expectation indicates the average value of the duration of the project. For a work fragment, the distribution will have the same features: a large difference between the most expected time period for the implementation of the work fragment (plans are traditionally based on it) and the average value of the term.
To predict the duration of the project, we determine the longest chain in time and add the duration of its fragments. If we add together the values distributed according to Gauss, then on average the correct final term should be obtained. After all, half of the fragments of the work are completed ahead of schedule, half later and these heterogeneities are mutually compensated. The more fragments, the more accurately the inhomogeneities are compensated. Plus, we can calculate the error and slightly shift the deadline by a sigma or two, depending on the consequences of the deadline.
But we do not have a Gaussian distribution. In our country, lengthening the execution time of each piece of work is much more likely than shortening it by the same amount. As a result, if we add the deadlines for the most probable completion of each piece of work, then the heterogeneities in terms of completion are not compensated, but amplified. Moreover, they are intensifying in the direction of lengthening the real average term of the project in comparison with the estimate. What we observe in real life: if the first term is overdue, then all other terms will be overdue, while the delay will only grow. Only by applying additional efforts can the project lag be stopped.
A feature of this method of calculation is a well-known fact: the smaller the fragmentation of work to assess the duration of the project, the less accurate the estimate. Although in theory it should be exactly the opposite. The reason for this is that our expert assessment for the entire work and for its constituent parts is taken from experience. Experience evaluates each element independently, and not based on arithmetic, according to which the duration of the work should be the sum of the durations of its constituent parts. Arithmetic does not work here, since the sum of the most probable deadlines for completing parts does not give the most probable deadline for completing all work - only mathematical expectations can be correctly summed.
If we try to slightly shift the estimated term for the entire project or its parts by sigma or two, considering the distribution to be normal, then this does not help much, since the distribution tail is much thicker than that of the Gaussian curve. As a result, due to such an increase in the estimate of the term, one cannot even reach the median, not to mention the average value of the duration of the project in the form of mathematical expectation.
On the one hand, mathematical expectations should be added. On the other hand, we are not mathematicians. We are from real life and we understand the problem of calculating the parameters of graphs even when there is time for this and some accumulated statistics. But there are other ways. In the end, the problem of assessing the timing of more than a dozen years, and the practice has learned to work with this.
Brooks method : we consider the implementation period of the program “head-on” (the user is the programmer himself) and multiply by 3 for the software product (the user is an unlimited circle of people) or the software complex (in the current realities - a set of microservices) and 9 for the system software product ( in current realities, a set of related infrastructure components). Where these coefficients come from is well theoretically justified in the first chapter of Brooks' book “Mythical Man-Month” and this description of 1975 is well shifted to current realities.
Scrum method : we introduce an abstract intermediate unit of the complexity of the implementation of the functional, we look at the statistics of the implementation of the functional measured in the given units of the functional, we ask the team to evaluate the complexity of the project tasks, translate the units into the Scrum iteration (sprints) of known length and get an estimate of the terms for this development team. Since we work with really collected statistics and the team is responsible for its estimates of the complexity, the binding of the duration to the unit of labor is a mathematical expectation, and therefore half of the sprints will end a little earlier, half a little later. In practice, in half of the Scrum iterations, the team will have to take part of the tasks, and in half add unplanned ones so that the sprint length remains constant.
The task of transferring the Scrum Evaluation of one team to another team is an art. They cannot be carried over by simply introducing the conversion factor of labor units of one team into labor units of another team. The fact is that one team has a good front-end, but it does not have a good base specialist, and the other team has a good base specialist, but the front-line tasks for it are of increased complexity. Differences in the specialization of developers will lead to the fact that with respect to backend tasks one team will have an excess in complexity towards tasks on the database, and the other on the front. In addition, the team itself determines the necessary level of the internal quality of the product and different teams determine it in a slightly different way than the neighboring teams, which also makes it difficult to recalculate labor units. That is, it is possible to translate the intermediate units of one team into the intermediate units of another team by comparing the estimates of similar tasks, but these tasks should be taken from different types of activities, taking into account the strengths and weaknesses of the teams and taking into account the peculiarities of setting up their Scrum.
Functional elements method : we introduce an intermediate unit of labor input, compile a list of functional elements (screens in a browser, microservices, custom infrastructure elements, etc.), estimate the laboriousness of working on each functional element in intermediate units. After that, based on our expert experience working with a specific development team, we evaluate the conversion factor of the intermediate unit over time. Personally, I still independently evaluate the conversion factors for different types of activities: analytics, designing, writing task statements, writing code, layout, testing, integration, etc. After this, the order of operations, the characteristic time delay between the completion of the previous operation and the start of a new one should be taken into account and the duration of the project should be determined by the critical path method.
So far, we have worked with the internal factors of the project. But we also have external ones: external contractors, related departments, suppliers and the client. The problems with them are exactly the same: they do twice as fast or respond much less often than one and a half times longer than the most probable value. That is, there is also no normal timing distribution. This should also be laid down and taken into account on the basis of statistics on work with clients, contractors and related units and, if possible, be protected with the help of penalties.
And so, we estimated the projected duration of the project. How to justify the received term? Based on the calculations made. For example, the author of a note for non-Scrum projects usually makes a large multi-line visual table in Google Sheets with a detailed calculation by the method of functional elements. The calculation is based on practice, all the coefficients and parameters are visual, and practice for interested parties is usually a strong and good argument, even if it turns out to be an uncomfortable period for certain individuals. Especially the practice is visual and obtained within the framework of the current company.
Will stakeholders, such as management, agree with a well-done and well-informed timeline assessment? Not always, even if the interested parties fully understand and realize that the assessment is correct. Sometimes the assessment is unacceptable for external reasons, but this is already a pain for the interested parties, resulting in a difficult decision by the management on the timing. Nevertheless, seeing the calculations based on experience, management and other interested parties are able to predictably influence the final term of the project by allocating additional resources and powers. Sometimes management who understands the timing situation will continue to cut deadlines without allocating additional resources and authority. How to live with this is the topic of a separate note.
This note explains the reason for the discrepancy between expert and theoretically calculated estimates. It is also considered why the “overstated” expert assessment is usually underestimated if it is not done on the basis of statistics on the implementation of similar projects. In the end, it is revealed how to correctly calculate the project term and explain the situation to interested parties before the start of the project or during the project.
The root of the error
When we evaluate the implementation period of a piece of work, we determine the most probable figure. It can be an expert assessment on one point or on three points, as in PERT. In a complex project, it can be parametric modeling. In all cases, we make the same mistake. The fact is that in all classical methods of estimation it is assumed that we have a normal distribution of the estimated value - according to Gauss. Theorists are very fond of this distribution.
Normal distribution means that a project with a correctly estimated duration of 6 months is equally likely to end in 9 months or 3 months. At equal distances from the distribution center, the probability of completion of the project is equal - this is a feature of the Gauss curve. On the other hand, in practice, few people saw an IT project that completed twice as fast (after 3 months), but projects that are delayed by one and a half times in terms (9 months) we see regularly. In addition, with a normal distribution, half of our projects would end before the estimated time, and half after. But this is not observed in practice either. This suggests that the normal distribution for estimating deadlines is not applicable. That is, we do not have a normal distribution, but some other, with different probability of completion before the most probable date and after.
Consider the lognormal distribution as an example of such a distribution. It will show us the features of this class of distributions. For the lognormal distribution, the median and mathematical expectation are significantly beyond the peak:
On the graph presented, the peak shows the greatest probability of the project completion date. This figure usually includes this figure plus a certain margin. The median indicates a separation point at which half of the projects will be completed before this deadline, and the second half after. Mathematical expectation indicates the average value of the duration of the project. For a work fragment, the distribution will have the same features: a large difference between the most expected time period for the implementation of the work fragment (plans are traditionally based on it) and the average value of the term.
To predict the duration of the project, we determine the longest chain in time and add the duration of its fragments. If we add together the values distributed according to Gauss, then on average the correct final term should be obtained. After all, half of the fragments of the work are completed ahead of schedule, half later and these heterogeneities are mutually compensated. The more fragments, the more accurately the inhomogeneities are compensated. Plus, we can calculate the error and slightly shift the deadline by a sigma or two, depending on the consequences of the deadline.
But we do not have a Gaussian distribution. In our country, lengthening the execution time of each piece of work is much more likely than shortening it by the same amount. As a result, if we add the deadlines for the most probable completion of each piece of work, then the heterogeneities in terms of completion are not compensated, but amplified. Moreover, they are intensifying in the direction of lengthening the real average term of the project in comparison with the estimate. What we observe in real life: if the first term is overdue, then all other terms will be overdue, while the delay will only grow. Only by applying additional efforts can the project lag be stopped.
A feature of this method of calculation is a well-known fact: the smaller the fragmentation of work to assess the duration of the project, the less accurate the estimate. Although in theory it should be exactly the opposite. The reason for this is that our expert assessment for the entire work and for its constituent parts is taken from experience. Experience evaluates each element independently, and not based on arithmetic, according to which the duration of the work should be the sum of the durations of its constituent parts. Arithmetic does not work here, since the sum of the most probable deadlines for completing parts does not give the most probable deadline for completing all work - only mathematical expectations can be correctly summed.
If we try to slightly shift the estimated term for the entire project or its parts by sigma or two, considering the distribution to be normal, then this does not help much, since the distribution tail is much thicker than that of the Gaussian curve. As a result, due to such an increase in the estimate of the term, one cannot even reach the median, not to mention the average value of the duration of the project in the form of mathematical expectation.
What to do
On the one hand, mathematical expectations should be added. On the other hand, we are not mathematicians. We are from real life and we understand the problem of calculating the parameters of graphs even when there is time for this and some accumulated statistics. But there are other ways. In the end, the problem of assessing the timing of more than a dozen years, and the practice has learned to work with this.
Brooks method : we consider the implementation period of the program “head-on” (the user is the programmer himself) and multiply by 3 for the software product (the user is an unlimited circle of people) or the software complex (in the current realities - a set of microservices) and 9 for the system software product ( in current realities, a set of related infrastructure components). Where these coefficients come from is well theoretically justified in the first chapter of Brooks' book “Mythical Man-Month” and this description of 1975 is well shifted to current realities.
Scrum method : we introduce an abstract intermediate unit of the complexity of the implementation of the functional, we look at the statistics of the implementation of the functional measured in the given units of the functional, we ask the team to evaluate the complexity of the project tasks, translate the units into the Scrum iteration (sprints) of known length and get an estimate of the terms for this development team. Since we work with really collected statistics and the team is responsible for its estimates of the complexity, the binding of the duration to the unit of labor is a mathematical expectation, and therefore half of the sprints will end a little earlier, half a little later. In practice, in half of the Scrum iterations, the team will have to take part of the tasks, and in half add unplanned ones so that the sprint length remains constant.
The task of transferring the Scrum Evaluation of one team to another team is an art. They cannot be carried over by simply introducing the conversion factor of labor units of one team into labor units of another team. The fact is that one team has a good front-end, but it does not have a good base specialist, and the other team has a good base specialist, but the front-line tasks for it are of increased complexity. Differences in the specialization of developers will lead to the fact that with respect to backend tasks one team will have an excess in complexity towards tasks on the database, and the other on the front. In addition, the team itself determines the necessary level of the internal quality of the product and different teams determine it in a slightly different way than the neighboring teams, which also makes it difficult to recalculate labor units. That is, it is possible to translate the intermediate units of one team into the intermediate units of another team by comparing the estimates of similar tasks, but these tasks should be taken from different types of activities, taking into account the strengths and weaknesses of the teams and taking into account the peculiarities of setting up their Scrum.
Functional elements method : we introduce an intermediate unit of labor input, compile a list of functional elements (screens in a browser, microservices, custom infrastructure elements, etc.), estimate the laboriousness of working on each functional element in intermediate units. After that, based on our expert experience working with a specific development team, we evaluate the conversion factor of the intermediate unit over time. Personally, I still independently evaluate the conversion factors for different types of activities: analytics, designing, writing task statements, writing code, layout, testing, integration, etc. After this, the order of operations, the characteristic time delay between the completion of the previous operation and the start of a new one should be taken into account and the duration of the project should be determined by the critical path method.
So far, we have worked with the internal factors of the project. But we also have external ones: external contractors, related departments, suppliers and the client. The problems with them are exactly the same: they do twice as fast or respond much less often than one and a half times longer than the most probable value. That is, there is also no normal timing distribution. This should also be laid down and taken into account on the basis of statistics on work with clients, contractors and related units and, if possible, be protected with the help of penalties.
Time justification
And so, we estimated the projected duration of the project. How to justify the received term? Based on the calculations made. For example, the author of a note for non-Scrum projects usually makes a large multi-line visual table in Google Sheets with a detailed calculation by the method of functional elements. The calculation is based on practice, all the coefficients and parameters are visual, and practice for interested parties is usually a strong and good argument, even if it turns out to be an uncomfortable period for certain individuals. Especially the practice is visual and obtained within the framework of the current company.
Will stakeholders, such as management, agree with a well-done and well-informed timeline assessment? Not always, even if the interested parties fully understand and realize that the assessment is correct. Sometimes the assessment is unacceptable for external reasons, but this is already a pain for the interested parties, resulting in a difficult decision by the management on the timing. Nevertheless, seeing the calculations based on experience, management and other interested parties are able to predictably influence the final term of the project by allocating additional resources and powers. Sometimes management who understands the timing situation will continue to cut deadlines without allocating additional resources and authority. How to live with this is the topic of a separate note.
All Articles