How QA organize test automation on a project. One practical way
Some time ago, I wrote an article about my experience in organizing the work of QA Engineer on a project. Now I want to continue this topic, but in a narrower direction - test automation. It will be about the same project , it is small, but developing according to the requests of regular customers. Perhaps my approach is not very suitable for teams with many dozens of employees and each is responsible for their part (in my opinion, everyone’s work in such projects should be strictly regulated, otherwise it’s simply impossible to manage such a colossus, although they will find a healthy grain), but he will certainly be interesting to those who, like me, once came to a new job, and stood at a crossroads how to organize their own place under the new sun.
So, how can you organize test automation on a project? If your answer is to take the wonderful Selenium and some kind of framework for it and on, then from the height of 13 years of experience in testing, I would not do so for sure. Will have to master new horizons.
What is fraught with the approach to "go in cycles in selenium"? And the fact that you have already read and heard more than once. The fact that these tests are UI, they are expensive and long both in writing and running, and in maintaining performance. Everyone knows this, but all the pain of this truth is encountered only when it is ignored. UI end-to-end autotests should be mandatory, no one except them will give you confidence that what the client sees after the next deployment will not be a disappointment for him. But they should not become the center of automation of your testing.
Take Test Pyramid Principles as your Quality Control Center. The same ones, about which a lot has already been said, but in practice, not every QA understands how to make this pyramid applied.
It is drawn like this:
And like this:
Everyone will find an option for their architecture.
Firstly, no matter how lazy and unusual it may be, master the white-skinned testing and become a “partially developer”.
Raise the project locally, just as every developer does (for sure, the project has documentation on how to do this, find it). You’ll probably curse this business a dozen times, doing it for the first time, you will have a million questions and problems, but when you do this you will already know a lot: what are the architectural ideas, what database, where from where are forwarded, where are the necessary configs , and most importantly, where autotests are stored and how to run them.
Secondly, deal with autotests written by the developers themselves.
Find their place in the project structure, what types of testing and tools are used, how they are launched. Evaluate the project’s coverage with autotests, learn how to navigate them. There are many tools for automatic coverage assessment, you can use it, but I'm not talking about that now. I’m more about the development of your intuition: take a piece of functionality and see how it is covered by autotests: good or bad. If it’s good, an occasion to be calm for this part, if it’s bad, you have found yourself a work front for those days when there are no urgent tasks on the agenda. In the meantime, keep in mind that every place in the code is vulnerable.
Thirdly, take courage, review the developers' pull request.
Every time you start developing a new functional, write down all the test cases for yourself. Check out those of them that need to be automated so as not to return to manual testing of this functionality. Review the pull requests of the developers and reasonably demand the implementation of the autotests that they missed. Many times I came across very cool developers who wrote cool tests, but even they yielded to a good QA in understanding how exactly the final client will most likely use the product. Therefore, even for seasoned specialists, there is something to recommend in writing autotests.
Fourth, be even more courageous and write autotests directly in the product code.
Yes, just like the developers do. On a par with them. Every time you come across a test case that needs to be automated, do not rush to immediately implement it with separate tools that are used only within the QA team. And first of all, ask yourself the question: can it be automated through unit / integration self-tests in the product code? If so, do just that. If it’s scary, then this is scary only for the first time, but what level-up of your skills, because experienced programmers will review your pull requests. Yes, at first everything that you have done will be blown to smithereens, but development always involves pain :) But in the end you will receive wonderful dividends:
Fifth, create a limited small set of UI end-to-end autotests that mimic the actions of an end user in a browser.
These will be tests that will only be supported by the QA team. They can be embodied in
And yes, you now have access to the project, and it costs you nothing to update the front-end of the project so as to have convenient and reliable locators for Selenium. No need to wait and ask for anyone - open pull requests in the main product code.
Now I am working on just such a scenario. As a result, at the top of my testing pyramid there are only 9 pieces of end-to-end tests. And their support is my responsibility. And all the other tens and hundreds of tests live together with the main product code, begin their work on the local computer of the developer and are supported by the entire team of engineers.
My end-to-end tests work for quite some time, due to the fact that they upload video files to the platform, and then convert them with different parameters, send them for processing to third-party services, wait for an answer, and so on, without any stubs. Therefore, in autotests there are many moments of waiting for the end of the operation. The team did not like the prospect of having such tests in CircleCi, and it doesn’t really need to. So I implemented them like job in Jenkins.
When all test checks in the product code have been completed, we deploy the tested brunch to the test environment and run end-to-end tests on it using Jenkins. A few more manual functional tests for greater fidelity from QA and that's it, you can merge the brunch in the master. Today, I’m not the only one who drives Jenkins for autotests on a test environment; developers constantly launch them when they need to generate fresh test data and simulate user activity. There are few of them, so they are stable and always work. Convenient to everyone.
I note one more pleasant advantage for the QA Engineer in such an implementation of the testing pyramid. It turns out, you become a part of a team of engineers in full. You really do an integral part of a single work - write code with everyone, look at the code of developers, communicate with them, and they do the same to you. You see each other's work, better collaboration, stronger team building. You will understand the project not only from the outside, but also from the inside, worthy of respect, will not it?
My article does not claim to be a universal pill that can be used anytime, anywhere. All projects are very different, all teams are very different - each should build its own best path itself, often this is the quintessence of different approaches and tools. Even the famous Scrum, how many projects I saw, each professes in his own way :) I don’t believe in strict instructions at all, I believe that they are needed as guidelines, but I need to act according to the situation.
However, the IT world is developing, and more and more people are coming into it, so I am sure that among the readers of this material there will certainly be someone to whom my little instructions will help me choose the right path. Smile at me in comments if it was useful :), feedback will be pleasant to me!
Getting ready to study
So, how can you organize test automation on a project? If your answer is to take the wonderful Selenium and some kind of framework for it and on, then from the height of 13 years of experience in testing, I would not do so for sure. Will have to master new horizons.
What is fraught with the approach to "go in cycles in selenium"? And the fact that you have already read and heard more than once. The fact that these tests are UI, they are expensive and long both in writing and running, and in maintaining performance. Everyone knows this, but all the pain of this truth is encountered only when it is ignored. UI end-to-end autotests should be mandatory, no one except them will give you confidence that what the client sees after the next deployment will not be a disappointment for him. But they should not become the center of automation of your testing.
Take Test Pyramid Principles as your Quality Control Center. The same ones, about which a lot has already been said, but in practice, not every QA understands how to make this pyramid applied.
It is drawn like this:
And like this:
Everyone will find an option for their architecture.
Putting Testing Pyramid Principles Into QA
Firstly, no matter how lazy and unusual it may be, master the white-skinned testing and become a “partially developer”.
Raise the project locally, just as every developer does (for sure, the project has documentation on how to do this, find it). You’ll probably curse this business a dozen times, doing it for the first time, you will have a million questions and problems, but when you do this you will already know a lot: what are the architectural ideas, what database, where from where are forwarded, where are the necessary configs , and most importantly, where autotests are stored and how to run them.
Secondly, deal with autotests written by the developers themselves.
Find their place in the project structure, what types of testing and tools are used, how they are launched. Evaluate the project’s coverage with autotests, learn how to navigate them. There are many tools for automatic coverage assessment, you can use it, but I'm not talking about that now. I’m more about the development of your intuition: take a piece of functionality and see how it is covered by autotests: good or bad. If it’s good, an occasion to be calm for this part, if it’s bad, you have found yourself a work front for those days when there are no urgent tasks on the agenda. In the meantime, keep in mind that every place in the code is vulnerable.
Thirdly, take courage, review the developers' pull request.
Every time you start developing a new functional, write down all the test cases for yourself. Check out those of them that need to be automated so as not to return to manual testing of this functionality. Review the pull requests of the developers and reasonably demand the implementation of the autotests that they missed. Many times I came across very cool developers who wrote cool tests, but even they yielded to a good QA in understanding how exactly the final client will most likely use the product. Therefore, even for seasoned specialists, there is something to recommend in writing autotests.
Fourth, be even more courageous and write autotests directly in the product code.
Yes, just like the developers do. On a par with them. Every time you come across a test case that needs to be automated, do not rush to immediately implement it with separate tools that are used only within the QA team. And first of all, ask yourself the question: can it be automated through unit / integration self-tests in the product code? If so, do just that. If it’s scary, then this is scary only for the first time, but what level-up of your skills, because experienced programmers will review your pull requests. Yes, at first everything that you have done will be blown to smithereens, but development always involves pain :) But in the end you will receive wonderful dividends:
- such tests will be supported by the entire development team
- they will work continuously, continuously and quickly
- they will be integrated into CircleCi and run every time they deploy automatically (if this is implemented on the project)
- holes will be patched in testing (you write what was not written before you)
- you can help developers write tests if they don’t have time
Fifth, create a limited small set of UI end-to-end autotests that mimic the actions of an end user in a browser.
These will be tests that will only be supported by the QA team. They can be embodied in
- The most popular, critical customer scenarios
- the fact that in integration tests in the main code it is not possible to fully implement, for example, work with third-party services
And yes, you now have access to the project, and it costs you nothing to update the front-end of the project so as to have convenient and reliable locators for Selenium. No need to wait and ask for anyone - open pull requests in the main product code.
What comes of it
Now I am working on just such a scenario. As a result, at the top of my testing pyramid there are only 9 pieces of end-to-end tests. And their support is my responsibility. And all the other tens and hundreds of tests live together with the main product code, begin their work on the local computer of the developer and are supported by the entire team of engineers.
My end-to-end tests work for quite some time, due to the fact that they upload video files to the platform, and then convert them with different parameters, send them for processing to third-party services, wait for an answer, and so on, without any stubs. Therefore, in autotests there are many moments of waiting for the end of the operation. The team did not like the prospect of having such tests in CircleCi, and it doesn’t really need to. So I implemented them like job in Jenkins.
When all test checks in the product code have been completed, we deploy the tested brunch to the test environment and run end-to-end tests on it using Jenkins. A few more manual functional tests for greater fidelity from QA and that's it, you can merge the brunch in the master. Today, I’m not the only one who drives Jenkins for autotests on a test environment; developers constantly launch them when they need to generate fresh test data and simulate user activity. There are few of them, so they are stable and always work. Convenient to everyone.
I note one more pleasant advantage for the QA Engineer in such an implementation of the testing pyramid. It turns out, you become a part of a team of engineers in full. You really do an integral part of a single work - write code with everyone, look at the code of developers, communicate with them, and they do the same to you. You see each other's work, better collaboration, stronger team building. You will understand the project not only from the outside, but also from the inside, worthy of respect, will not it?
Final farewell
My article does not claim to be a universal pill that can be used anytime, anywhere. All projects are very different, all teams are very different - each should build its own best path itself, often this is the quintessence of different approaches and tools. Even the famous Scrum, how many projects I saw, each professes in his own way :) I don’t believe in strict instructions at all, I believe that they are needed as guidelines, but I need to act according to the situation.
However, the IT world is developing, and more and more people are coming into it, so I am sure that among the readers of this material there will certainly be someone to whom my little instructions will help me choose the right path. Smile at me in comments if it was useful :), feedback will be pleasant to me!
All Articles