Good day. I am doing test automation. Like all automation engineers, I have a set of libraries and tools that I usually choose to write tests. But periodically there are situations when none of the familiar libraries can solve the problem with the risk of making autotests unstable or fragile. In this article, I would like to tell you how the seemingly standard task of using mock'ov led me to write my module. I would also like to share my decision and hear feedback.

application

One of the necessary sectors in the financial sector is audit. Data needs to be checked regularly (reconciliation). In this regard, the application that I tested appeared. In order not to talk about something abstract, let's imagine that our team is developing an application for processing applications from instant messengers. For each application, an appropriate event must be created in elasticsearch. The verification application will be our monitoring that applications are not skipped.

So, let's imagine that we have a system that has the following components:

1. Configuration server. For the user, this is a single entry point, where he configures not only the application for verification, but also other components of the system.
2. Verification application.
3. Data from the application processing applications that are stored in elasticsearch.
4. Reference data. The data format depends on the messenger with which the application is integrated.

Task

Test automation in this case looks quite straightforward:

1. Environment Preparation:
   
   
   • Installs elasticsearch with minimal configuration (using msi and command line).
   • A verification application is installed.
2. Test execution:
   
   
   • A verification application is configured.
   • Elasticsearch is filled with test data for the corresponding test (how many applications were processed).
   • The application receives "reference" data from the messenger (how many applications were supposedly actually).
   • The verdict issued by the application is checked: the number of successfully verified applications, the number of missing applications, etc.
3. Cleaning the environment.

The problem is that we are testing monitoring, but to configure it, we need data from the configuration server. Firstly, installing and configuring a server for each run is a time-consuming operation (it has its own base, for example). Secondly, I want to isolate applications in order to simplify the localization of problems when finding a defect. In the end, it was decided to use mock.

This may raise the question: "If we still mock the server, maybe we can’t spend time installing and filling elasticsearch, but replace mock?". But still, you should always remember that the use of mock provides flexibility, but adds an obligation to monitor the relevance of mock's behavior. Therefore, I refused to replace elasticsearch: it is quite easy to install and fill it.

First mock

The server sends the configuration to GET requests in several ways in / configuration. We are interested in two ways. The first is /configuration/data_cluster



with cluster configuration

 { "host": "127.0.0.1", "port": 443, "credentials": { "username": "user", "password": "pass" } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The second is /configuration/reconciliation



with the configuration of the drilling application

 { "reconciliation_interval": 3600, "configuration_update_interval": 60, "source": { "address": "file:///c:/path", "credentials": { "username": "user", "password": "pass" } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The difficulty is that you need to be able to change the server response during the test or between tests in order to test how the application responds to configuration changes, incorrect passwords, etc.

So, static mocks and tools for mocks in unit tests (mock, monkeypatch from pytest, etc.) will not work for us. I found a great pretenders



library that I thought was right for me. Pretenders provides the ability to create an HTTP server with rules that determine how the server will respond to requests. Rules are stored in presets, which allows you to isolate mocks for different test suites. Presets can be cleared and re-filled, allowing you to update answers as needed. It is enough to raise the server itself once during the preparation of the environment:

 python -m pretenders.server.server --host 127.0.0.1 --port 8000
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And in the tests we need to add the use of the client. In the simplest case, when the answers are completely hardcoded in the tests, it may look like this:

 import json import pytest from pretenders.client.http import HTTPMock from pretenders.common.constants import FOREVER @pytest.fixture def configuration_server_mock(request): mock = HTTPMock(host="127.0.0.1", port=8000, name="server") request.addfinalizer(mock.reset) return mock def test_something(configuration_server_mock): configuration_server_mock.when("GET /configuration/data_cluster").reply( headers={"Content-Type": "application/json"}, body=json.dumps({ "host": "127.0.0.1", "port": 443, "credentials": { "username": "user", "password": "pass", }, }), status=200, times=FOREVER, ) configuration_server_mock.when("GET /configuration/reconciliation").reply( headers={"Content-Type": "application/json"}, body=json.dumps({ "reconciliation_interval": 3600, "configuration_update_interval": 60, "source": { "address": "file:///c:/path", "credentials": { "username": "user", "password": "pass", }, }, }), status=200, times=FOREVER, ) # test application
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But that's not all. With its flexibility, pretenders



has two limitations that must be remembered and must be addressed in our case:

1. Rules cannot be deleted one at a time. In order to change the answer, you must delete the entire preset and recreate all the rules again.
2. All paths used in the rules are relative. Presets have a unique path of the form / mockhttp / <preset_name>, and this path is a common prefix for all created paths in the rules. The tested application receives only the host name, and it cannot know about the prefix.

The first limitation is very unpleasant, but can be solved by writing a module that encapsulates the work with the configuration. For example, so

 configuration.data_cluster.port = 443
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

or (to make update requests less frequently)

 data_cluster_config = get_default_data_cluster_config() data_cluster_config.port = 443 configuration.update_data_cluster(data_cluster_config)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Such encapsulation allows us to update all paths almost painlessly. You can also create an individual preset for each individual endpoint and a general (main) preset, redirecting (through 307 or 308) to individual ones. Then you can only clear one preset to update the rule.

In order to get rid of prefixes, you can use the mitmproxy library. This is a powerful tool that allows, among other things, to redirect requests. We will remove the prefixes as follows:

 mitmdump --mode reverse:http://127.0.0.1:8000 --replacements :~http:^/:/mockhttp/server/ --listen-host 127.0.01 --listen-port 80
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The parameters of this command do the following:

1. --listen-host 127.0.0.1
   
   
   
   and --listen-port 80
   
   
   
   obvious. Mitmproxy raises its server, and with these parameters we determine the interface and port that this server will listen to.
2. --mode reverse:http://127.0.0.1:8000
   
   
   
   means that requests to the mitproxy server will be redirected to http://127.0.0.1:8000
   
   
   
   . Read more here .
3. --replacements :~http:^/:/mockhttp/server/
   
   
   
   defines a template by which requests will be changed. It consists of three parts: a request filter ( ~http
   
   
   
   for HTTP requests), a template for changing ( ^/
   
   
   
   to replace the beginning of the path), and actually replacing ( /mockhttp/server
   
   
   
   ). Read more here .

In our case, we add mockhttp/server



to all HTTP requests and redirect them to http://127.0.0.1:8000



, i.e. to our server pretenders. As a result, we have achieved that now the configuration can be obtained with a GET request to http://127.0.0.1/configuration/data_cluster



.

In general, I was satisfied with the design with pretenders



and mitmproxy



. With the apparent complexity - after all, 2 servers instead of one real one - preparation consists in installing 2 packages and executing 2 commands on the command line to start them. Not everything is so simple in managing a mock, but all the complexity lies in only one place (managing presets) and is solved quite simply and reliably. However, new circumstances appeared in the problem that made me think about a new solution.

Second mock

Until that moment, I almost did not say where the reference data came from. An attentive reader might notice that in the example above, the path to the file system is used as the address of the data source. And it really does work like that, but only for one of the vendors. Another vendor provides an API for receiving applications, and it was with him that a problem arose. It’s difficult to raise the vendor’s API during tests, so I planned to replace it with mock according to the same scheme as before. But to receive applications, a request of the form

 GET /application-history?page=2&size=5&start=1569148012&end=1569148446
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

There are 2 points here. Firstly, a few options. The fact is that parameters can be specified in any order, which greatly complicates the regular expression for the pretenders



rule. It is also necessary to remember that the parameters are optional, but this is not such a problem as random order. Secondly, the last parameters (start and end) specify the time interval for filtering the order. And the problem in this case is that we can not predict in advance which interval (not the magnitude, but the start time) will be used by the application to form the mock response. Simply put, we need to know and use parameter values ​​to form a “reasonable” answer. “Reasonability” in this case is important, for example, so that we can test that the application goes through all pages of the pagination: if we answer all the requests the same way, then we can’t find defects due to the fact that only one page out of five is requested .

I tried to look for alternative solutions, but in the end I decided to try to write my own. So there was a loose server . This is a Flask application in which paths and responses can be configured after it starts. Out of the box, he knows how to work with rules for the type of request (GET, POST, etc.) and for the path. This allows you to replace pretenders



and mitmproxy



in the original task. I will also show how it can be used to create a mock for the vendor API.

An application needs 2 main paths:

1. Base endpoint. This is the same prefix that will be used for all configured rules.
2. Configuration endpoint. This is the prefix of those requests with which you can configure the mock server itself.

 python -m looseserver.default.server.run --host 127.0.0.1 --port 80 --base-endpoint / --configuration-endpoint /_mock_configuration/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

In general, it is best not to configure the base endpoint and configuration endpoint so that one is the parent of the other. Otherwise, there is a risk that the paths for configuration and for testing will conflict. Configuration endpoint will take precedence, since Flask rules are added for configuration earlier than for dynamic paths. In our case, we could use --base-endpoint /configuration/



if we were not going to include the vendor API in this mock.

The simplest version of the tests does not change much

 import json import pytest from looseserver.default.client.http import HTTPClient from looseserver.default.client.rule import PathRule from looseserver.default.client.response import FixedResponse @pytest.fixture def configuration_server_mock(request): class MockFactory: def __init__(self): self._client = HTTPClient(configuration_url="http://127.0.0.1/_mock_configuration/") self._rule_ids = [] def create_rule(self, path, json_response): rule = self._client.create_rule(PathRule(path=path)) self._rule_ids.append(rule.rule_id) response = FixedResponse( headers={"Content-Type": "application/json"}, status=200, body=json.dumps(json_response), ) self._client.set_response(rule_id=rule.rule_id, response=response) def _delete_rules(self): for rule_id in self._rule_ids: self._client.remove_rule(rule_id=rule_id) mock = MockFactory() request.addfinalizer(mock._delete_rules) return mock def test_something(configuration_server_mock): configuration_server_mock.create_rule( path="configuration/data_cluster", json_response={ "host": "127.0.0.1", "port": 443, "credentials": { "username": "user", "password": "pass", }, } ) configuration_server_mock.create_rule( path="configuration/reconciliation", json_response={ "reconciliation_interval": 3600, "configuration_update_interval": 60, "source": { "address": "file:///applications", "credentials": { "username": "user", "password": "pass", }, }, } )
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Fixture has become more difficult, but the rules can now be deleted one at a time, which simplifies the work with them. Using mitmproxy



no longer needed.

Let's go back to the vendor API. We will create a new type of rules for a loose server, which, depending on the value of the parameter, will give different answers. Next we will use this rule for the page parameter.

New rules and answers need to be created for both the server and the client. Let's start with the server:

 from looseserver.server.rule import ServerRule class ServerParameterRule(ServerRule): def __init__(self, parameter_name, parameter_value=None, rule_type="PARAMETER"): super(ServerParameterRule, self).__init__(rule_type=rule_type) self._parameter_name = parameter_name self._parameter_value = parameter_value def is_match_found(self, request): if self._parameter_value is None: return self._parameter_name not in request.args return request.args.get(self._parameter_name) == self._parameter_value
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Each rule must define an is_match_found



method, which determines whether it should work for a given request or not. The input parameter for it is the request object. After the new rule is created, it is necessary to "teach" the server to accept it from the client. To do this, use RuleFactory



:

 from looseserver.default.server.rule import create_rule_factory from looseserver.default.server.application import configure_application def _create_application(base_endpoint, configuration_endpoint): server_rule_factory = create_rule_factory(base_endpoint) def _parse_param_rule(rule_type, parameters): return ServerParameterRule( rule_type=rule_type, parameter_name=parameters["parameter_name"], parameter_value=parameters["parameter_value"], ) server_rule_factory.register_rule( rule_type="PARAMETER", parser=_parse_param_rule, serializer=lambda rule_type, rule: None, ) return configure_application( rule_factory=server_rule_factory, base_endpoint=base_endpoint, configuration_endpoint=configuration_endpoint, ) if __name__ == "__main__": application = _create_application(base_endpoint="/", configuration_endpoint="/_mock_configuration") application.run(host="127.0.0.1", port=80)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Here we create a factory for default rules so that it contains the rules that we used before and register a new type. In this case, the client does not need the rule information, so the serializer



actually does nothing. Further, this factory is transferred to the application. And it can already be run like a regular Flask application.

The situation with the client is similar: we create a rule and a factory. But for the client, firstly, it is not necessary to define the is_match_found



method, and secondly, the serializer in this case is necessary to send the rule to the server.

 from looseserver.client.rule import ClientRule from looseserver.default.client.rule import create_rule_factory class ClientParameterRule(ClientRule): def __init__(self, parameter_name, parameter_value=None, rule_type="PARAMETER", rule_id=None): super(ClientParameterRule, self).__init__(rule_type=rule_type, rule_id=rule_id) self.parameter_name = parameter_name self.parameter_value = parameter_value def _create_client(configuration_url): def _serialize_param_rule(rule): return { "parameter_name": rule.parameter_name, "parameter_value": rule.parameter_value, } client_rule_factory = create_rule_factory() client_rule_factory.register_rule( rule_type="PARAMETER", parser=lambda rule_type, parameters: ClientParameterRule(rule_type=rule_type, parameter_name=None), serializer=_serialize_param_rule, ) return HTTPClient(configuration_url=configuration_url, rule_factory=client_rule_factory)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

It remains to use _create_client



to create the client, and the rules can be used in tests. In the example below, I added the use of another default rule: CompositeRule



. It allows you to combine several rules into one so that they work only if each of them returns True for calling is_match_found



.

 @pytest.fixture def configuration_server_mock(request): class MockFactory: def __init__(self): self._client = _create_client("http://127.0.0.1/_mock_configuration/") self._rule_ids = [] def create_paged_rule(self, path, page, json_response): rule_prototype = CompositeRule( children=[ PathRule(path=path), ClientParameterRule(parameter_name="page", parameter_value=page), ] ) rule = self._client.create_rule(rule_prototype) self._rule_ids.append(rule.rule_id) response = FixedResponse( headers={"Content-Type": "application/json"}, status=200, body=json.dumps(json_response), ) self._client.set_response(rule_id=rule.rule_id, response=response) ... mock = MockFactory() request.addfinalizer(mock._delete_rules) return mock def test_something(configuration_server_mock): ... configuration_server_mock.create_paged_rule( path="application-history", page=None, json_response=["1", "2", "3"], ) configuration_server_mock.create_paged_rule( path="application-history", page="1", json_response=["1", "2", "3"], ) configuration_server_mock.create_paged_rule( path="application-history", page="2", json_response=["4", "5"], )
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Conclusion

A mitmproxy



pretenders



and mitmproxy



provides a powerful and flexible enough tool for creating mocks. Its advantages:

1. Easy setup.
2. Ability to isolate query sets using presets.
3. Deletes an entire isolated set at once.

By cons include:

1. The need to create regular expressions for rules.
2. The inability to change the rules individually.
3. The presence of a prefix for all created paths or the use of redirection using mitmproxy
   
   
   
   .

Documentation links:

Pretenders

Mitmproxy

Loose server