Testing and project economics

In my work I constantly use unit tests. And you? In my experience, most programmers are very rare. When interviewing candidates for vacancies in my team, I always ask the question: "Do you have any testing experience?" And most often I hear in response: "No." And if you ask why, then the most common answer will be: "The customer did not let me do this."



This approach surprises me. Will you tell the builder which brand of concrete to choose for building a house? Or auto mechanics - which part of the engine to repair, and which not, if the engine has flashed in the car? It is unlikely, because these are serious technical issues, the solution of which must be entrusted to a professional. A professional has the necessary skills and tools that allow him to solve a problem faster and cheaper.



Why count the money



The Civil Code has the concept of “saving a contractor through experience and knowledge”. Its meaning is that the work is performed by a professional who knows how to optimize the process, how to do better, but at the same time cheaper. And developers as professionals must give the customer the best solutions for the least money.



But actually, programmers are focused on something else. Many of them are sure that software development is a purely technical process, which, of course, has its own value, but the programmer's job is to write, not to count money. However, in my opinion, development is primarily an economic project, which is backed by money and calculations, and only after that is a technical task.



The logic here is simple: the business - the very customers who order us software development for their companies - in fact, do not need software at all. Business needs money, the benefits this software will bring.



This means that programmers can not do without understanding the development economics, knowing the cost of technical solutions and which of them are optimal in terms of not only technology, but also finance.



Software development is generally not the cheapest part of the business, but poor-quality software can lead to big losses. Even downtime leads to lost profits, and software problems can lead to millions of losses. Therefore, the developer as a professional must be confident in the quality of his product and guarantee this to his client. But the quality of the product is the most important indicator. And the customer must know that this quality can be measured and be confident in it. And how exactly is the programmer’s task.



I, like many of my colleagues, believe that programmers should not discuss engineering solutions with the customer. It is enough in general to know what is important for the client, what functionality and for what purposes it needs. Which decisions to choose is the responsibility of the programmer. And in the framework of this responsibility, he must use the best engineering practices for a given project with economically justified costs. Therefore, professionals who know how to make the development process cheaper win.



Which test is more profitable



One practice that allows you to control product quality is testing. But which test is more profitable? To answer this question, it is enough to compare several parameters. All tests can be conditionally divided not only by type of technology, but also by speed, and most importantly - by cost.



Unit test - testing production units. By units I mean program units, depending on the language represented by classes, functions, less often files. Using the TDD approach, testing is initially woven into the product development process. A project can use thousands of such tests to test individual pieces of code. They have an excellent indicator of speed - the entire set of tests can be completed in seconds. These are the cheapest tests.



Integration tests - testing 2x-3x or more units together. There can be hundreds or thousands of such tests on a project, depending on the programmer. Speed ​​- seconds or minutes for the entire set of tests. Cost is higher than unit tests.



Acceptance tests simulate user actions with a program. They require the preparation of an operational environment; therefore, they are complex and expensive. Quantity per project - usually one test is done on a business story. The speed of work is usually from tens of minutes to several hours for the entire set of tests.



The most expensive and difficult is manual testing . You need to hire a person, train him, create and give him a business history map so that he tests new software on it. Such testing takes several days for the entire set of tests.



Any type of test requires preparation, as well as a certain investment of money. And in order to get savings (and therefore benefits), we must first take into account the speed of testing and the complexity of its launch. If the project does not have automatic tests, then there is only one way out - manual testing, the most expensive and slow.



However, the modern approach to software development is a continuous cycle: in a week new functionality or a new version is created and checked automatically or manually. That is, the product changes frequently, and all these changes must be controlled so that the programmer is sure that the new and old functionalities work well. Therefore, manual testing loses immediately on all indicators.



As for the different types of automatic testing, there are three more important aspects to consider.



First of all, it is reliability - how easy it is to break the test (this is the best indicator for unit tests).



Then the environment that is needed to run the tests. Unit tests are very simple, they themselves create an environment around the unit under test. Integration tests need a service that must be installed, running, etc. Acceptance tests require preparation, since the application must work in its environment.



And finally - coverage (test coverage). Unit tests have good coverage and are used throughout the project. Integration tests are usually used on certain parts of the project, that is, they do not completely cover the program. The coverage of acceptance tests is worse - this is due to the complexity of writing and running them, most often they cover the main business cases, this is 20-40% of the total project volume.



Conclusion



So, it turns out that unit tests are leading in all indicators of this brief comparative analysis. In my practice, most often I use them, it gives sufficient quality. If the quality requirements increase, then we move on to acceptance. The use of integration tests depends on the customer, who may want to test certain parts of the project.



My current technology stack is angular and .net. And if on the server side testing does not raise big questions, then testing the client application is still, in many places, not obvious to everyone. On my client projects, the number of unit tests varies from a few hundred to several thousand. In the next article I will try to share key techniques for testing angular applications.



All Articles