Overclocking Magento Rest API with RoadRunner
PHP created to die. And everything would be fine, but he has not been given a chance to do this recently. A year ago, on the hub, the announcement of the RoadRunner tool took place , forcing the PHP process to leave the endless circle of death and resurrection.
The principle of RoadRunner’s work is to keep the running process and throw incoming requests into it, which allows, according to the developers, to increase application performance (sometimes even 40 times).
Since I have been working with Magento for a long time, it seemed like a great idea to test the tool not on a mythical framework, but on a real application, for which Magento Open Source was a great fit.
Initialization Cost of Magento Application
The way to speed up the RoadRunner application involves reducing the response time (after a warm-up start) by reducing the overhead of initializing the application.
Screenshot from Anton Tsitou's presentation "Designing hybrid Go / PHP applications using RoadRunner"
In the case of Magento, the main time spent on startup falls on:
- composer autoloading
- bootstrapping
Composer autoloading does not attract attention, since it is standard for a PHP application.
Profiling results related to Composer.
Magento application bootstraping includes initializing an error handler, checking application status, etc.
The hardest part is initializing the IoC container (“ObjectManager” in terms of Magento) and recursively instantiating dependencies through it to get the application object.
Profiling results related to bootstraping.
RoadRunner Implementation
To start RoadRunner, you need to create a worker that will contain a cycle for accepting incoming requests and sending responses. Moreover, the tool works with requests and answers implementing PSR-7. From the official documentation, it looks something like this:
while ($req = $psr7->acceptRequest()) { $resp = new \Zend\Diactoros\Response(); $resp->getBody()->write("hello world"); $psr7->respond($resp); }
Magento and PSR-7
Magento has not yet implemented the PSR-7 and out of the box uses its own request and response implementations, the approaches to which are mostly dragged from the previous version.
To implement RoadRunner, you need to find an entry point that would accept the request in some form and return a response ( Symfony example ).
There is such a point in Magento , \ Magento \ Framework \ AppInterface , there is only one problem, this interface is not designed to accept the request. But wait, where does it get into the application from? It is worth going back to the beginning and the mantra - PHP is born to die . Accordingly, the vast majority of libraries, packages, frameworks, when designing and dividing into layers, simply do not assume that the request, it turns out, is not just one global.
The Magento transport layer is built on the same principle. Although the documentation describes the differences between injectable / newable objects , in reality we have the use of the request as a global statefull service, initializing itself from global variables ($ _GET, $ _POST). In addition to all this, the injection of this service can be seen at all levels of the application in the kernel itself, let alone the quality of third-party modules.
Based on the foregoing, the hope of implementing RoadRunner only through the conversion of requests from PSR-7-style to Magento-style was spent.
Implement PSR-7 adapter
We formulate the problem, taking into account the information received.
I would like to have a certain application interface that accepts a PSR-7 request and returns a PSR-7 response. It is also necessary to create an implementation of the created interface that adapts this interaction format to the Magento application.
PSR-7 adapter
As mentioned above, magento application already returns a response, so we only need to convert it to PSR-7 format.
For the request, you need a class that proxies all calls to the current request object, which we put in a special register (let the gods of architecture forgive this perversion). In addition, it was found that the request class is not used one, but 3, so it requires you to bind them all through the IoC configuration of the container.
The set of classes used to work with queries in Magento
Problems of an undying PHP application
An application launched through RoadRunner has the same problems as any long-lived php process, they are described in the documentation ( https://roadrunner.dev/docs/usage-production )
The main problems of the application level that the developer should monitor are:
- memory leaks ( https://www.php.net/manual/ru/features.gc.php )
- database connections (suddenly) can fall off by timeout, like any other
- competent state management
Particular attention in the context of Magento should be given to state management, since both in the kernel and in third-party modules, caching the current user / product / category inside the service is a very common approach.
protected function getCustomer(): ?CustomerInterface { if (!$this->customer) { if ($this->customerSession->isLoggedIn()) { $this->customer = $this->customerRepository->getById($this->customerSession->getCustomerId()); } else { return null; } } return $this->customer; }
An example of a method from the kernel using the state of an object.
Launching the Magento Rest API Server via RoadRunner
Given the potential problems with the global state, based on the experience of developing the front-end part of Magento, the most suitable and painless WebApi part was chosen for launch.
The first thing to do is to create our worker who will run through RoadRunner and live endlessly (almost). To do this, take a piece of code from the RoadRunner guides and add our application there, wrapped in a PSR-7 adapter.
$relay = new StreamRelay(STDIN, STDOUT); $psr7 = new PSR7Client(new Worker($relay)); $bootstrap = \Magento\Framework\App\Bootstrap::create(BP, []); /** @var \Magento\Framework\App\Http $app */ $app = $bootstrap->createApplication(\Magento\Framework\App\Http::class); /** @var ApplicationInterface $psr7Application */ $psr7Application = $bootstrap->getObjectManager()->create( \Isxam\M2RoadRunner\Application\MagentoAppWrapper::class, [ 'magentoApp' => $app ] ); while ($request = $psr7->acceptRequest()) { try { $response = $psr7Application->handle($request); $psr7->respond($response); } catch (\Throwable $e) { $psr7->getWorker()->error((string)$e); } }
The code before the while loop will be executed at the start of the worker, all that is inside the loop - with every new request.
When our worker is ready, we proceed to configure the RoadRunner server written in Go. Everything is nice and fast here, only the composer package that downloads the executable file doesn’t need to be installed, which cannot be pleasant. We create the configuration of our server - the simplest one looks something like this.
http: address: 0.0.0.0:8086 workers: command: "php worker.php" pool: numWorkers: 1
RoadRunner configuration.
The documentation contains an abundance of settings that allow you to flexibly configure the server so that the desire to compile your binary does not exactly come.
./rr serve -v -d
Server start
Solution testing
Instruments
For convenient testing, we take something simple, for example artillery.io.
We will test the performance with the help of one user executing queries sequentially (RoadRunner also supports query execution in several threads, we will leave this question to other researchers)
At the input, we have the artillery config file with two environments - Apache and RoadRunner. They both work with the same Magento instance, so here they are on an equal footing.
Test scenarios
The following scenarios were used to measure the performance of the two solutions.
- name: "S1. Create category" flow: - loop: - post: url: "/rest/V1/categories" json: category: name: "name-{{prefix}}-{{ $loopCount }}" parent_id: 2 is_active: true count: 100
- name: "S2. Countries list" flow: - loop: - get: url: "/rest/V1/directory/countries" count: 100
- name: "S3. Product types list" flow: - loop: - get: url: "/rest/V1/products/types" count: 100
- name: "S4. Product attribute sets list" flow: - loop: - get: url: "/rest/V1/products/attribute-sets/sets/list?searchCriteria" count: 100
- name: "S5. Category get" flow: - loop: - get: url: "/rest/V1/categories/2" count: 100
- name: "S6. Create product" flow: - loop: - post: url: '/rest/V1/products' json: product: sku: "sku-{{prefix}}-{{ $loopCount }}" name: "name-{{prefix}}-{{ $loopCount }}" attribute_set_id: 4 price: 100 type_id: "simple" count: 100
- name: "S7. Get product list" flow: - loop: - get: url: "/rest/V1/products?searchCriteria[pageSize]=20" count: 100
Result
After running all the scripts alternately through RoadRunner and Apache, medians of the duration of the query were obtained. According to the medians, the speed of all scenarios differs by approximately the same value of ~ 50ms.
Performance Test Result.
Total
A hands-on experiment confirmed the assumptions about the constancy of the RoadRunner performance gain on a particular application. Using this tool allows you to speed up the processing of application requests for a constant equal to the environment initialization time and dependencies.
On light handlers, this allows you to speed up the application at times, on heavy it almost does not give a tangible effect. If your code is slow, then most likely the plantain will not help him.
If your application is well written, then most likely there will be no problems with its operation through RoadRunner, but if the application requires adaptation for use with RoadRunner, then most likely it would have required the same without RoadRunner in order to more clearly observe the layers of architecture and following development standards in the field.
Magento Open Source is generally suitable for launching in the above environment, however, it requires improvements to the transport layer and correcting the business logic to prevent incorrect behavior during repeated requests within the same process. Also, the use of RoadRunner imposes certain restrictions on development approaches, but they do not contradict established practices.
References
All Articles