JMeter - Swiss Tester Knife (Part 2)
Today I’ll tell you how, with the help of JMeter, we established the process of caching product pages, tested the operation of a mobile application without the application itself and created 2,000 users in the system without access to the database.
Whoever doesn’t know what is going on here, read the first part of the article - in it I told how JMeter helped us “catch” the problem with the fall of the site among some users.
Another of our clients is an Italian company manufacturing equipment for indoor, outdoor and decorative lighting.
The equipment of our clients illuminates the masterpieces of world painting, architectural monuments, monuments, buildings of business centers and airports
We developed the site for this company on Kentico. It presents all the materials and models of lighting equipment - 20,000 copies. And each has a separate page.
After the release of the site, we discovered one unpleasant feature of Kentico: it does not have a tool for automatically loading media files into the cache. After updating and restarting the site (which happens regularly), the cache is reset and most pages open for 10 seconds. Not every user is ready to wait so much.
We saw two options for loading media files into the cache. First: it’s a long time to manually go through the site manually through a browser. Second: write a script that will do this for us - it fits!
We decided to use JMeter: we just fed him the full sitemap, he walked on it and sent the media files to the cache.
It was a simple solution, but not very optimal. The script took six (!!!) hours to complete. It needed some faster way.
Having studied the topic a bit, we decided to use regexps. JMeter can ask the server for resources built into the page for certain masks. This we took advantage of. Having eliminated what is not located on our server and does not require loading into the cache (third-party images, fonts, styles, etc.), we have reduced the time it takes for the script to go through the pages to two hours.
This is what the HTTP Request in JMeter should look like to load only a certain type of asset
But this is too long: the filling starts at 9:00 in Chelyabinsk and takes an hour, then another two hours are spent on the script, and as a result we will finish at 12:00. In Italy, it is 08:00, and some locals start working at 6-7 a.m. The site is slow, complaints are pouring ... We need to optimize further!
The decision came when we realized that almost the same images were used on all 20,000 product pages. We carried out the analysis and realized that it is enough to load completely (with all the built-in resources) only 30% of random pages, and the rest can be run without loading media files. Thus, the vast majority of materials - pictures, fonts, scripts and styles for product pages - fall into the cache. When a user contacts one of them, downloading from the server will require only a small part of the content, and it will take no more than 1-2 seconds.
By requesting a random page number from the User Defined Variables table, each time we get a random page from the list
Using the described method, we got a site that in 90% of cases works quickly. Yes, in one case out of ten it took up to 30 seconds to wait for the page to load, but in the other nine everything just flew. And that’s in just 15-30 minutes.
Now such tricks are no longer needed - the developers have optimized the caching system itself. But at that time, this decision was a real salvation, and JMeter turned out to be a very useful tool, even after release and pouring.
The third client, which I will talk about, is a company that develops, manufactures and sells medical equipment: from simple scalpels to laser cutters and complex drills with built-in cameras. Their managers go to clinics, communicate with surgeons and tell them about their products. To do this, they have a special application for the iPad: the doctor’s data and a list of tools that interest him are entered into it. Later, a link to a personal page with a list of all these tools, detailed descriptions and characteristics comes to the doctor’s email. It was these pages for surgeons that we developed. The application and email newsletter was made by another company.
One of the tasks for JMeter on this project is to test the system for creating different versions of such pages, despite the fact that we had only a backend. We wrote a script so that using a random number generator it would create pages of various types, and then send them to our working emails for verification. Everything is simple. And I would not even mention this client in the article, if not for one case.
A few days before the launch, my manager called me and asked to create 2000 users for the application. The difficulty was that we did not have access to either the live server or the database. We had to create 2000 users only through the CMS (in this case, through Umbraco). At the same time, after registration, each user should receive a letter of the following type:
Upon receiving such an email, the user installs the application and changes the default password to his own
To register a user, five fields are required:
I had only two hours to complete the task. Once again: 2000 users, five fields for each of them, two hours ... Data from the manager came in the following form:
If I were a developer, and not a QA specialist, I would write a script, for example, in perl: he would parse this data and quickly scatter everything. But I chose a different path and used the most, perhaps, the ideal tool for fast data organization - MS Excel. Spreadsheets save the world!
With the help of simple manipulations with replacing data, cutting off excess and other tricks, I created a file of ~ 10k lines, of the form:
login1 | john.smith
login2 | doctor.who
...
name1 | John
...
I didn’t bother with passwords much - I just wrote PasswordNumber1 and “extended” this column in Excel to all 2000 users. This does not matter, because it was used only for the first entrance. Yes, and I did not sort the general list of emails alphabetically, which means that the probability of random password guessing was minimized.
Feeding this file (via the clipboard) to JMeter, I leaned back in my chair and watched with pleasure how users were created. The whole process of processing data and creating new users took 1 hour 45 minutes - even time left for coffee!
Conclusion: JMeter turned out to be a convenient tool for automated testing of the backend of mobile applications, and also, suddenly, for automation of non-standard tasks - like this one, with a bunch of users and a minimum of information about them.
In general, JMeter can be used (and was used by me personally) for a variety of tasks. For example, for:
From the foregoing, we can draw a general conclusion: JMeter is a universal tool, a real Swiss knife for a tester (and for a developer, to be honest).
Whoever doesn’t know what is going on here, read the first part of the article - in it I told how JMeter helped us “catch” the problem with the fall of the site among some users.
How to quickly cache a couple of thousand product pages
Another of our clients is an Italian company manufacturing equipment for indoor, outdoor and decorative lighting.
The equipment of our clients illuminates the masterpieces of world painting, architectural monuments, monuments, buildings of business centers and airports
We developed the site for this company on Kentico. It presents all the materials and models of lighting equipment - 20,000 copies. And each has a separate page.
After the release of the site, we discovered one unpleasant feature of Kentico: it does not have a tool for automatically loading media files into the cache. After updating and restarting the site (which happens regularly), the cache is reset and most pages open for 10 seconds. Not every user is ready to wait so much.
We saw two options for loading media files into the cache. First: it’s a long time to manually go through the site manually through a browser. Second: write a script that will do this for us - it fits!
We decided to use JMeter: we just fed him the full sitemap, he walked on it and sent the media files to the cache.
It was a simple solution, but not very optimal. The script took six (!!!) hours to complete. It needed some faster way.
Having studied the topic a bit, we decided to use regexps. JMeter can ask the server for resources built into the page for certain masks. This we took advantage of. Having eliminated what is not located on our server and does not require loading into the cache (third-party images, fonts, styles, etc.), we have reduced the time it takes for the script to go through the pages to two hours.
This is what the HTTP Request in JMeter should look like to load only a certain type of asset
But this is too long: the filling starts at 9:00 in Chelyabinsk and takes an hour, then another two hours are spent on the script, and as a result we will finish at 12:00. In Italy, it is 08:00, and some locals start working at 6-7 a.m. The site is slow, complaints are pouring ... We need to optimize further!
The decision came when we realized that almost the same images were used on all 20,000 product pages. We carried out the analysis and realized that it is enough to load completely (with all the built-in resources) only 30% of random pages, and the rest can be run without loading media files. Thus, the vast majority of materials - pictures, fonts, scripts and styles for product pages - fall into the cache. When a user contacts one of them, downloading from the server will require only a small part of the content, and it will take no more than 1-2 seconds.
By requesting a random page number from the User Defined Variables table, each time we get a random page from the list
Using the described method, we got a site that in 90% of cases works quickly. Yes, in one case out of ten it took up to 30 seconds to wait for the page to load, but in the other nine everything just flew. And that’s in just 15-30 minutes.
Now such tricks are no longer needed - the developers have optimized the caching system itself. But at that time, this decision was a real salvation, and JMeter turned out to be a very useful tool, even after release and pouring.
How to create 2000 application users, if you only have their e-mail and two hours to work
The third client, which I will talk about, is a company that develops, manufactures and sells medical equipment: from simple scalpels to laser cutters and complex drills with built-in cameras. Their managers go to clinics, communicate with surgeons and tell them about their products. To do this, they have a special application for the iPad: the doctor’s data and a list of tools that interest him are entered into it. Later, a link to a personal page with a list of all these tools, detailed descriptions and characteristics comes to the doctor’s email. It was these pages for surgeons that we developed. The application and email newsletter was made by another company.
One of the tasks for JMeter on this project is to test the system for creating different versions of such pages, despite the fact that we had only a backend. We wrote a script so that using a random number generator it would create pages of various types, and then send them to our working emails for verification. Everything is simple. And I would not even mention this client in the article, if not for one case.
A few days before the launch, my manager called me and asked to create 2000 users for the application. The difficulty was that we did not have access to either the live server or the database. We had to create 2000 users only through the CMS (in this case, through Umbraco). At the same time, after registration, each user should receive a letter of the following type:
Upon receiving such an email, the user installs the application and changes the default password to his own
To register a user, five fields are required:
- name;
- surname;
- login;
- password.
I had only two hours to complete the task. Once again: 2000 users, five fields for each of them, two hours ... Data from the manager came in the following form:
If I were a developer, and not a QA specialist, I would write a script, for example, in perl: he would parse this data and quickly scatter everything. But I chose a different path and used the most, perhaps, the ideal tool for fast data organization - MS Excel. Spreadsheets save the world!
With the help of simple manipulations with replacing data, cutting off excess and other tricks, I created a file of ~ 10k lines, of the form:
login1 | john.smith
login2 | doctor.who
...
name1 | John
...
I didn’t bother with passwords much - I just wrote PasswordNumber1 and “extended” this column in Excel to all 2000 users. This does not matter, because it was used only for the first entrance. Yes, and I did not sort the general list of emails alphabetically, which means that the probability of random password guessing was minimized.
Feeding this file (via the clipboard) to JMeter, I leaned back in my chair and watched with pleasure how users were created. The whole process of processing data and creating new users took 1 hour 45 minutes - even time left for coffee!
Conclusion: JMeter turned out to be a convenient tool for automated testing of the backend of mobile applications, and also, suddenly, for automation of non-standard tasks - like this one, with a bunch of users and a minimum of information about them.
In general, JMeter can be used (and was used by me personally) for a variety of tasks. For example, for:
- optimization analysis: we conduct stress testing, roll optimization fixes, conduct testing again, compare the results;
- Query security test: we write the script using a proxy, send the request directly to the server without front-end checks, remove the various keys from the request, verify that the server responds adequately;
- checking the response to headers: we write a script that sends requests to the same page with different headers in a loop, we check the server’s reaction and see what should be removed;
- enumerating authorization combinations: put a cycle with passwords in a cycle with logins, test the authorization form;
- checks for protection against attacks: we send requests for submitting forms in several threads and see if your protection works;
- automation of API systems: any results of API requests can be processed by a script and used in the next step of the test cycle - this way you can automate testing of any such system.
From the foregoing, we can draw a general conclusion: JMeter is a universal tool, a real Swiss knife for a tester (and for a developer, to be honest).
All Articles