The book "Microservices. Development and refactoring patterns »

Hello, habrozhiteli! If for a long time it seems to you that all development and deployment in your company has slowed down to the utmost - go to the microservice architecture. It provides continuous development, delivery and deployment of applications of any complexity.



The book, intended for developers and architects from large corporations, tells how to design and write applications in the spirit of microservice architecture. It also describes how to refactor a large application - and the monolith turns into a set of microservices.



We offer you to read the passage "Transaction Management in Microservice Architecture"



Almost any request processed by an industrial application is executed as part of a database transaction. Developers of such applications use frameworks and libraries that simplify working with transactions. Some tools provide an imperative API for manually starting, committing, and rolling back transactions. And frameworks like Spring have a declarative mechanism. Spring supports the @Transactional annotation, which automatically invokes a method within a transaction. Thanks to this, writing transactional business logic becomes quite simple.



To be more precise, managing transactions is simple in monolithic applications that access a single database. If the application uses several databases and message brokers, this process becomes more difficult. Well, in the microservice architecture, transactions cover several services, each of which has its own database. In such circumstances, the application should use a more sophisticated mechanism for working with transactions. As you will soon see, the traditional approach to distributed transactions is not viable in modern applications. Microservice-based systems should use storytelling instead.



But before moving on to the narratives, let's see why transaction management creates so many complexities in a microservice architecture.

4.1.1. Microservice architecture and the need for distributed transactions

Imagine that you are a developer at FTGO and are responsible for implementing the createOrder () system operation. As described in Chapter 2, this operation should ensure that the customer can place orders, check order details, authorize the customer’s bank card, and create an Order record in the database. The implementation of these actions would be relatively simple in a monolithic application. All the data necessary to verify the order is ready and available. In addition, ACID transactions could be used to ensure data consistency. You could simply specify the @Transactional annotation for the createOrder () service method.



However, performing this operation in a microservice architecture is much more difficult. As seen in fig. 4.1, the data required by createOrder () operations is scattered across several services. createOrder () reads information from the Consumer service and updates the contents of the Order, Kitchen, and Accounting services.



Since each service has its own database, you should use a mechanism to coordinate data between them.

4.1.2. Distributed Transaction Issues

The traditional approach to ensuring data consistency between several services, databases or message brokers is the use of distributed transactions. The de facto standard for distributed transaction management is X / Open XA (see en.wikipedia.org/wiki/XA ). The XA model uses two-phase commit (2PC) to ensure that all changes to the transaction are saved or rolled back. This requires that the databases, message brokers, database drivers, and messaging APIs comply with the XA standard, and an interprocess communication mechanism that distributes global XA transaction identifiers is also required. Most relational databases are XA compatible, as are some message brokers. For example, a Java EE-based application can perform distributed transactions using the JTA.









Despite their simplicity, distributed transactions have a number of problems. Many modern technologies, including NoSQL databases such as MongoDB and Cassandra, do not support them. Distributed transactions are not supported by some modern message brokers like RabbitMQ and Apache Kafka. So, if you decide to use distributed transactions, many modern tools will not be available to you.



Another problem with distributed transactions is that they are a form of synchronous IPC, which impairs availability. In order for a distributed transaction to be committed, all services involved in it must be accessible. As described in Chapter 3, system accessibility is a product of the accessibility of all participants in a transaction. If two services with 99.5% availability participate in a distributed transaction, the overall availability will be 99%, which is much less. Each additional service lowers the availability. Eric Brewer has formulated the CAP theorem, which states that a system can have only two of the following three properties: consistency, accessibility, and partition resistance (en.wikipedia.org/wiki/CAP_ Theorem). Today, architects favor affordable systems, sacrificing consistency.



At first glance, distributed transactions may seem attractive. From the developer's point of view, they have the same software model as local transactions. But due to the problems described earlier, this technology is not viable in modern applications. Chapter 3 showed how to send messages as part of a database transaction without using distributed transactions. To solve the more complex problem of ensuring data consistency in a microservice architecture, the application must use another mechanism based on the concept of loosely coupled asynchronous services. And here the narratives come in handy.

4.1.3. Use the Storytelling template to maintain data consistency

Narratives are a mechanism for ensuring data consistency in a microservice architecture without the use of distributed transactions. The narrative is created for each system team that needs to update data in several services. This is a sequence of local transactions, each of which updates data in one service, using the familiar frameworks and libraries for ACID transactions mentioned earlier.

Storytelling Template



Ensures data consistency between services using a sequence of local transactions that are coordinated using asynchronous messages. See microservices.io/patterns/data/saga.html .

The system operation initiates the first stage of the narration. The completion of one local transaction leads to the following. In section 4.2, you will see how the coordination of these steps is accomplished using asynchronous messages. An important advantage of asynchronous messaging is that it ensures that all stages of the narration are completed, even if one or more participants are inaccessible.



Narrations have several important differences from ACID transactions. First of all, they lack isolation (for more on this, see Section 4.3). In addition, since each local transaction captures its changes, to roll back the story, you must use compensating transactions, which we will discuss later in this section. Consider a storytelling example.

Narrative example: creating an order

In this chapter, we use the Create Order narrative as an example (Fig. 4.2). It implements the createOrder () operation. The first local transaction is triggered by an external order creation request. The remaining five transactions are triggered one after the other.









This narrative consists of the following local transactions.



1. Order service. Creates an order with state APPROVAL_PENDING.



2. Service Consumer. Checks if the customer can place orders.



3. Kitchen service. Checks the details of the order and creates a request with the status CREATE_PENDING.



4. Accounting service. Authorizes the customer’s bank card.



5. Kitchen service. Changes the status of an application to AWAITING_ACCEPTANCE.



6. Service Order. Changes the order status to APPROVED.



In Section 4.2, I will show how the services involved in the story interact with each other using asynchronous messages. The service publishes a message upon completion of a local transaction. This initiates the next stage of the narration and allows not only to achieve a weak cohesion of the participants, but also to guarantee the full implementation of the narration. Even if the recipient is temporarily unavailable, the broker buffers the message until it can be delivered.



The narratives seem simple, but their use is associated with some difficulties, especially with a lack of isolation between them. The solution to the problem is described in section 4.3. Another non-trivial aspect is the rollback of changes when an error occurs. Let's see how this is done.

Narratives use offsetting transactions to roll back changes

Traditional ACID transactions have one great feature: business logic can easily roll back a transaction if a business rule violation is detected. It simply executes the ROLLBACK command, and the database discards all changes made so far. Unfortunately, the story cannot be rolled back automatically, because at each stage it captures the changes in the local database. This, for example, means that in the event of a failed authorization of a bank card in the fourth stage of the Create Order narrative, the FTGO application must manually cancel the changes made in the previous three stages. You must write the so-called offset transactions.



Suppose the (n + 1) -th transaction in the story failed. It is necessary to neutralize the consequences of the previous n transactions. At a conceptual level, each of these Ti stages has its own compensating transaction Ci, which cancels the effect of Ti. To compensate for the effect of the first n stages, the narrative must execute each transaction Ci in the reverse order. The sequence looks like this: T1 ... Tn, Cn ... C1 (Fig. 4.3). In this example, step Tn + 1 fails, which requires the cancellation of steps T1 ... Tn.









The narrative performs compensatory transactions in the reverse order to the original ones: Cn ... C1. Here, the same sequential execution mechanism operates as in the case of Ti. The termination of Ci should initiate Ci - 1.



Take, for example, the narrative of Create Order. It may fail for a variety of reasons.



1. Incorrect information about the customer, or the customer is not allowed to create orders.



2. Incorrect information about the restaurant, or the restaurant is not able to accept the order.



3. Inability to authorize a customer’s bank card.



In the event of a failure in the local transaction, the storytelling coordination mechanism must perform compensatory steps that reject the order and possibly the order. In the table. 4.1 compensating transactions are collected for each stage of the Create Order narrative. It should be noted that not every stage requires a compensating transaction. This applies, for example, to read operations, such as verifyConsumerDetails (), or to the authorizeCreditCard () operation, all steps after which always succeed.









In Section 4.3, you will learn that the first three stages of the Create Order narrative are called transactions that are available for compensation, because the steps following them may fail. The fourth step is called a turning transaction because the next steps never fail. The last two steps are called repeatable transactions, because they always end successfully.



To understand how compensatory transactions are used, imagine a situation in which authorization of a customer’s bank card fails. In this case, the narrative performs the following local transactions.



1. Order service. Creates an order with state APPROVAL_PENDING.



2. Service Consumer. Checks if the customer can place orders.



3. Kitchen service. Checks the details of the order and creates a request with the status CREATE_PENDING.



4. Accounting service. Makes an unsuccessful attempt to authorize a customer's bank card.



5. Kitchen service. Changes the status of the application to CREATE_REJECTED.



6. Service Order. Changes the order status to REJECTED.



The fifth and sixth stages are compensatory transactions that cancel the updates made by the Kitchen services and, accordingly, Order. The coordinating narrative logic is responsible for the sequence of direct and compensating transactions. Let's see how it works.

about the author

Chris Richardson is the developer, architect, and author of POJOs in Action (Manning, 2006), which describes how to build enterprise-level Java applications using the Spring and Hibernate frameworks. He bears the honorary titles of Java Champion and JavaOne Rock Star.



Chris developed the original version of CloudFoundry.com, an early implementation of the Java PaaS platform for Amazon EC2.



Nowadays, he is considered a recognized ideological leader in the world of microservices and regularly speaks at international conferences. Chris created microservices.io, which contains microservice design patterns. He also conducts consultations and trainings around the world for organizations that are switching to microservice architecture. Chris is currently working on his third startup, Eventuate.io. It is a software platform for developing transactional microservices.



»More details on the book can be found on the publisher’s website

» Contents

» Excerpt



25% off coupon for hawkers - Microservice Patterns

Upon payment of the paper version of the book, an electronic book is sent by e-mail.