Kaspresso: the autotest framework you were waiting for
Everyone who has been or is engaged in Android autotests knows what a pain it is.
You get tired of the volume of tasks and problems so that even a vacation does not help. People even quit because of autotests.
Pain, suffering and torment inevitably lead to the emergence of something new and beautiful. We tried to put together all the rakes that we had to step on, joined forces with the guys from Avito and HH and created something that would make your relationship with auto-tests incomparably better and more fruitful.
Meet: Kaspresso - the autotest framework you were waiting for!
I immediately note that there are two fairly high-quality videos on the network dedicated to Kaspresso and AdbServer (auxiliary for Kaspresso, but at the same time, a strong and independent project):
1. Dmitry Movchan, Eugene Matsyuk - How to start writing autotests and not go crazy
2. Egor Kournikov - The only thing you need for UI testing
They describe in detail the history of the creation of libraries: how we moved from solving one problem to solving another, and what happened in the end. I highly recommend it.
Therefore, in the article I will only talk about the main points and chips so that you can quickly understand what this framework is all about and what tasks it solves. And guts and details can be found in the video and documentation.
Every developer who starts writing autotests inevitably asks questions:
To all this add that each team is trying to solve these issues somehow in its own way. As a result, there is no single answer to them, there is nowhere to peek how to do it right. Therefore, writing and supporting autotests is expensive for companies, which calls into question the appropriateness of automation as such.
Meanwhile, test automation pursues good goals. It allows you to always have a green and ready-to-release wizard. And this means that you can roll out the release quickly.
The only problem is finding answers to the above questions. But they are almost the same for all teams. So why not automate their solution?
Let's reveal our expectations from the framework a little.
By default, only the Espresso library is available to us in Android. There are, of course, Appium + Cucumber, which in theory allow you to write tests on two platforms at once. But the community is confidently moving towards the very first instrument. I will not describe all the pros and cons of the above libraries: the network is full of information about this. Here, for example, is one of the relatively recent links: Appium vs Espresso. What to choose and how to use?
So, Espresso. Not a bad tool, but its api is a bit turned inside out. Take a look at a simple code:
You have to think to figure it out, right?
The gods from Thailand and Australia presented us with the Kakao library, with which you can understand at a glance what is happening in your test. Here is the same code, but with Kakao:
Much better. But now imagine that you have automated a whole test case. What would the code be like?
The main problem is that, looking at this test, it is difficult to correlate it with the test case that you automated. You can, of course, add logs or something like that. Or enter dsl, which immediately transforms the look of your tests:
Agree, it looks completely different.
Any library for ui-tests fails. The same action can be performed 50 times successfully, and on the 51st break for no apparent reason. And on the 52nd run, everything is fine again. And such "flacking" can decently spoil your nerves.
We thought, why not try to intercept all the actions of Kakao-Espresso, and already add additional behavior there aimed at handling such random errors.
That is how version 2.1 of the Kakao library was born, allowing you to integrate into all Espresso calls.
In addition, we created our own interceptors, with which you can change the behavior at the dial-peer or, for example, simply log. Moreover, these interceptors are customizable, so you can customize them to fit your needs. Read more in the dock .
Specifically, as part of the fight against flaky-tests - if some of your actions threw an exception, then Kaspresso will try:
With these steps, we completely solve the issue with flaky tests!
One of the main problems that accompanies autotests is the lack of intelligible logging, which is especially not enough when the test crashes. Sit and wonder what and how has fallen here.
Thanks to the aforementioned intercepting, we were able to build a fairly extensive logging system.
Let's take a look at an example:
By default, Kaspresso logs all your actions, selects each step of the test, displays the execution time, etc.
Initially, adb is not available in Espresso tests. Yes, there is adb shell , but there are much fewer functions than in full adb . But there are so many things in it that can come in handy in tests.
We have created a separate AdbServer library that will return full adb to your tests! The above videos detail how we fought and what we went through for it ( one and two ).
The specifics of the tests at Kaspersky Lab is such that we have to work a lot with OS Android: set up some settings, upload files to the system, etc. All this prompted us to standardize all our work with the system, creating a set of clear interfaces, accessible through a single entry point - class Device .
What are these interfaces and what are they doing? Let me give you a couple of slides from Yegor's presentation as an illustration:
The documentation is here .
Under the hood, AdbServer and UiAutomator are mainly used.
But! If you suddenly are not satisfied with the implementation of an interface, you can specify your implementation through the Configurator.
All projects where there is localization, often there is a need to take screenshots in different languages to give to the translator as illustrations. It is very difficult to make a correct translation without seeing where and how a particular line is used. Therefore, I want to be able to take screenshots quickly and immediately in all languages. Including screenshots of system dialogs. You may also need screenshots on legacy screens, and without global refactoring.
All this allows you to make Kaspresso out of the box. Read more in the documentation .
One of the key tasks of Kaspresso was the creation of such a dsl, which would push you to the correct architecture of tests and their correct writing.
Many copies were broken on this topic, because you, unfortunately, will not find such rules anywhere. The most you can find are articles on Page Object .
Therefore, we did not spare our efforts and highlighted these issues both in the documentation and in the video once and video two .
In addition, Sasha Blinov wrote an excellent article about the Kotlin DSL and elegant tests . The dsl described in the article are provided by Kaspresso.
Back at Mobius, we proposed an option on how to accelerate the impact of autotests and quickly integrate them into PullRequest, bypassing the inevitable infrastructure problems. We talk more about this here .
The main charm is that if you already have many tests written in Kakao and you want to implement Kaspresso, then you do not need to rewrite anything! Just inherit your test classes from the special TestCase class. And that’s all!
It was:
It became:
And if you don't like inheritance, use a similar TestRule class.
As we already mentioned, Kaspresso is a very flexible and customizable framework. All settings are available through the Kaspresso class of the same name.
The default setting is default. If you want to customize something, it will look something like this:
That is, through TestCase constructor Kaspresso.Builder is available, where you set all the settings you need. Details about the configurator are written in the documentation .
In the very near future we plan to add the following things:
Through a special interceptor, we prepare data for Marathon . This allows us to see Allure reports of the following nature:
Details in PR # 4
PS PR is already in version 1.0.1, now we are preparing the corresponding PR in Marathon .
PSS There is an idea to attach a specific piece of the log to each step, and add a screenshot to the “fallen” step.
Often it is necessary to check the correctness of application upgrades. Yes, some of the checks can be transferred to unit tests. But we would like to be calm for the whole application as a whole.
Unfortunately, on a pure Espresso it is impossible to do this, because if we reinstall the tested apk, the test will fail. You can somehow try to trick with the runner, but it's hard for me to imagine how such improvements will look and how stable they will be.
Therefore, at Kaspresso, we are preparing a solution to this problem, based on UiAutomator. However, at the top you will have all the same familiar dsl sticking out, very similar to Kakao and with the same support for intersepting.
Kaspresso
AdbServer
Chat , in which we will always be happy to answer all your questions
Special thanks to everyone who participated in the development of the project.
It was very difficult, but damn cool!
We believe that Kaspresso and AdbServer will make your life better.
We welcome your feedback, recommendations, Yishuyam and PulRequest!
And do not forget to put an asterisk, please!
PS And at the very end a small poll =)
You get tired of the volume of tasks and problems so that even a vacation does not help. People even quit because of autotests.
Pain, suffering and torment inevitably lead to the emergence of something new and beautiful. We tried to put together all the rakes that we had to step on, joined forces with the guys from Avito and HH and created something that would make your relationship with auto-tests incomparably better and more fruitful.
Meet: Kaspresso - the autotest framework you were waiting for!
I immediately note that there are two fairly high-quality videos on the network dedicated to Kaspresso and AdbServer (auxiliary for Kaspresso, but at the same time, a strong and independent project):
1. Dmitry Movchan, Eugene Matsyuk - How to start writing autotests and not go crazy
2. Egor Kournikov - The only thing you need for UI testing
They describe in detail the history of the creation of libraries: how we moved from solving one problem to solving another, and what happened in the end. I highly recommend it.
Therefore, in the article I will only talk about the main points and chips so that you can quickly understand what this framework is all about and what tasks it solves. And guts and details can be found in the video and documentation.
Why do we even need our own framework?
Every developer who starts writing autotests inevitably asks questions:
- How to start writing autotests?
- What tools to choose?
- What to do if there is no necessary tool?
- What are the best practices?
To all this add that each team is trying to solve these issues somehow in its own way. As a result, there is no single answer to them, there is nowhere to peek how to do it right. Therefore, writing and supporting autotests is expensive for companies, which calls into question the appropriateness of automation as such.
Meanwhile, test automation pursues good goals. It allows you to always have a green and ready-to-release wizard. And this means that you can roll out the release quickly.
The only problem is finding answers to the above questions. But they are almost the same for all teams. So why not automate their solution?
What do we want from the framework?
Let's reveal our expectations from the framework a little.
Good readability
By default, only the Espresso library is available to us in Android. There are, of course, Appium + Cucumber, which in theory allow you to write tests on two platforms at once. But the community is confidently moving towards the very first instrument. I will not describe all the pros and cons of the above libraries: the network is full of information about this. Here, for example, is one of the relatively recent links: Appium vs Espresso. What to choose and how to use?
So, Espresso. Not a bad tool, but its api is a bit turned inside out. Take a look at a simple code:
@Test fun espressoTest() { onView(allOf(allOf(withId(R.id.espresso), isDescendantOfA(withId(R.id.coffee_variates))), isDescendantOfA(withId(R.id.content)))) .check(matches(withEffectiveVisibility(View.VISIBLE))) }
You have to think to figure it out, right?
The gods from Thailand and Australia presented us with the Kakao library, with which you can understand at a glance what is happening in your test. Here is the same code, but with Kakao:
@Test fun kakaoTest() { mainScreen { myView.isVisible() } }
Much better. But now imagine that you have automated a whole test case. What would the code be like?
@RunWith(AndroidJUnit4::class) class OpenHomeScreenTest: TestCase() { @Test fun test() { MainScreen() { homeButton.click() } HomeScreen() { title { isVisible() hasAnyText() } } } }
The main problem is that, looking at this test, it is difficult to correlate it with the test case that you automated. You can, of course, add logs or something like that. Or enter dsl, which immediately transforms the look of your tests:
@RunWith(AndroidJUnit4::class) class OpenHomeScreenTest: TestCase() { @Test fun test() { step(“1. Open Home screen”) { MainScreen() { homeButton.click() } } step(“2. Check Home title”) { HomeScreen() { title { isVisible() hasAnyText() } } } } }
Agree, it looks completely different.
Stability
Any library for ui-tests fails. The same action can be performed 50 times successfully, and on the 51st break for no apparent reason. And on the 52nd run, everything is fine again. And such "flacking" can decently spoil your nerves.
We thought, why not try to intercept all the actions of Kakao-Espresso, and already add additional behavior there aimed at handling such random errors.
That is how version 2.1 of the Kakao library was born, allowing you to integrate into all Espresso calls.
In addition, we created our own interceptors, with which you can change the behavior at the dial-peer or, for example, simply log. Moreover, these interceptors are customizable, so you can customize them to fit your needs. Read more in the dock .
Specifically, as part of the fight against flaky-tests - if some of your actions threw an exception, then Kaspresso will try:
- Scroll. Maybe your view is simply not visible on the screen.
- Remove the system dialogue that could come from God knows where.
- Repeat a broken call for two seconds.
With these steps, we completely solve the issue with flaky tests!
Logging
One of the main problems that accompanies autotests is the lack of intelligible logging, which is especially not enough when the test crashes. Sit and wonder what and how has fallen here.
Thanks to the aforementioned intercepting, we were able to build a fairly extensive logging system.
Let's take a look at an example:
By default, Kaspresso logs all your actions, selects each step of the test, displays the execution time, etc.
Full Adb in Espresso Tests
Initially, adb is not available in Espresso tests. Yes, there is adb shell , but there are much fewer functions than in full adb . But there are so many things in it that can come in handy in tests.
We have created a separate AdbServer library that will return full adb to your tests! The above videos detail how we fought and what we went through for it ( one and two ).
Work with Android OS
The specifics of the tests at Kaspersky Lab is such that we have to work a lot with OS Android: set up some settings, upload files to the system, etc. All this prompted us to standardize all our work with the system, creating a set of clear interfaces, accessible through a single entry point - class Device .
What are these interfaces and what are they doing? Let me give you a couple of slides from Yegor's presentation as an illustration:
The documentation is here .
Under the hood, AdbServer and UiAutomator are mainly used.
But! If you suddenly are not satisfied with the implementation of an interface, you can specify your implementation through the Configurator.
Screenshot for DocLoc (Documentation and Localization)
All projects where there is localization, often there is a need to take screenshots in different languages to give to the translator as illustrations. It is very difficult to make a correct translation without seeing where and how a particular line is used. Therefore, I want to be able to take screenshots quickly and immediately in all languages. Including screenshots of system dialogs. You may also need screenshots on legacy screens, and without global refactoring.
All this allows you to make Kaspresso out of the box. Read more in the documentation .
Architecture and best practices
One of the key tasks of Kaspresso was the creation of such a dsl, which would push you to the correct architecture of tests and their correct writing.
Many copies were broken on this topic, because you, unfortunately, will not find such rules anywhere. The most you can find are articles on Page Object .
Therefore, we did not spare our efforts and highlighted these issues both in the documentation and in the video once and video two .
In addition, Sasha Blinov wrote an excellent article about the Kotlin DSL and elegant tests . The dsl described in the article are provided by Kaspresso.
Back at Mobius, we proposed an option on how to accelerate the impact of autotests and quickly integrate them into PullRequest, bypassing the inevitable infrastructure problems. We talk more about this here .
How to connect and configure Kaspresso if you already have many tests
The main charm is that if you already have many tests written in Kakao and you want to implement Kaspresso, then you do not need to rewrite anything! Just inherit your test classes from the special TestCase class. And that’s all!
It was:
@RunWith(AndroidJUnit4::class) class OpenHomeScreenTest { private val mainScreen = MainScreen() private val homeScreen = HomeScreen() @get:Rule val activityTestRule = ActivityTestRule(MainActivity::class.java, true, false) @Test fun test() { ... } }
It became:
@RunWith(AndroidJUnit4::class) class OpenHomeScreenTest : TestCase() { private val mainScreen = MainScreen() private val homeScreen = HomeScreen() @get:Rule val activityTestRule = ActivityTestRule(MainActivity::class.java, true, false) @Test fun test() { ... } }
And if you don't like inheritance, use a similar TestRule class.
As we already mentioned, Kaspresso is a very flexible and customizable framework. All settings are available through the Kaspresso class of the same name.
The default setting is default. If you want to customize something, it will look something like this:
@RunWith(AndroidJUnit4::class) class OpenHomeScreenTest : TestCase( Kaspresso.Builder.default().apply { viewBehaviorInterceptors.add(MyInterceptor()) flakySafetyParams.timeoutMs = 1_000 } ) { private val mainScreen = MainScreen() private val homeScreen = HomeScreen() @get:Rule val activityTestRule = ActivityTestRule(MainActivity::class.java, true, false) @Test fun test() { ... } }
That is, through TestCase constructor Kaspresso.Builder is available, where you set all the settings you need. Details about the configurator are written in the documentation .
Immediate plans
In the very near future we plan to add the following things:
Display test steps in Allure (hello to guys from HeadHunter)
Through a special interceptor, we prepare data for Marathon . This allows us to see Allure reports of the following nature:
Details in PR # 4
PS PR is already in version 1.0.1, now we are preparing the corresponding PR in Marathon .
PSS There is an idea to attach a specific piece of the log to each step, and add a screenshot to the “fallen” step.
Testing upgrade scripts
Often it is necessary to check the correctness of application upgrades. Yes, some of the checks can be transferred to unit tests. But we would like to be calm for the whole application as a whole.
Unfortunately, on a pure Espresso it is impossible to do this, because if we reinstall the tested apk, the test will fail. You can somehow try to trick with the runner, but it's hard for me to imagine how such improvements will look and how stable they will be.
Therefore, at Kaspresso, we are preparing a solution to this problem, based on UiAutomator. However, at the top you will have all the same familiar dsl sticking out, very similar to Kakao and with the same support for intersepting.
useful links
Kaspresso
AdbServer
Chat , in which we will always be happy to answer all your questions
Acknowledgments
Special thanks to everyone who participated in the development of the project.
It was very difficult, but damn cool!
Instead of a conclusion
We believe that Kaspresso and AdbServer will make your life better.
We welcome your feedback, recommendations, Yishuyam and PulRequest!
And do not forget to put an asterisk, please!
PS And at the very end a small poll =)
All Articles