1, Concept of distributed transaction

1. What is a transaction

A transaction can be regarded as a big activity. It consists of different small activities. These activities either succeed or fail.

2. Local affairs

Four characteristics of database transactions ACID:

• A(Atomic):   Atomicity  ， All operations constituting a transaction are either completed or not executed at all. It is impossible to have partial success and partial failure.
• C(Consistency):   uniformity  ， Before and after the transaction, the consistency constraint of the database is not broken. For example, if Zhang San transfers 100 yuan to Li Si, the data before and after the transfer is correct, which is called consistency. If Zhang San transfers 100 yuan out and Li Si's account does not increase 100 yuan, there is a data error and consistency is not achieved.
• I(Isolation):   Isolation  ， Transactions in the database are generally concurrent. Isolation means that the execution of two concurrent transactions does not interfere with each other. One transaction cannot see the intermediate state of the running process of other transactions. By configuring the transaction isolation level, problems such as dirty reads and duplicate reads can be avoided.
• D(Durability):   persistence  ， After the transaction is completed, the data changes made by the transaction will be persisted to the database and will not be rolled back.

When implementing a database transaction, all operations involved in a transaction will be incorporated into an inseparable execution unit. All operations in the execution unit will either succeed or fail. As long as any operation fails, the whole transaction will roll back.

3. Distributed transaction

A distributed system will split an application system into multiple services that can be deployed independently, so it requires remote cooperation between services to complete transaction operations. In this distributed system environment, transactions completed by remote cooperation between different services through the network are called distributed transactions, such as user registration and delivery of points, creation of orders and inventory reduction, Bank transfer transactions are distributed transactions.

4. Scenario of distributed transaction generation

• A typical scenario is the microservice architecture, where microservices pass through each other   Remote call complete transaction   Operation. For example, order microservices and inventory microservices request inventory microservices to reduce inventory while placing an order. in short:   Generate distributed transactions across JVM processes  .
• A single system accesses multiple database instances,   Generate distributed transactions across database instances  .
• Multiple services access the same database instance  ， For example, order microservices and inventory microservices will generate distributed transactions even if they access the same database   Cross JVM processes  ， Two microservices hold different database links for database operations, and distributed transactions are generated at this time.

2, Basic theory of distributed transaction

1. CAP theory

a) Concept

CAP is the abbreviation of Consistency, Availability and Partition tolerance, which means Consistency, Availability and Partition tolerance respectively.

b) Combination mode

In all distributed transaction scenarios, the three characteristics of CAP will not be available at the same time, because C and A cannot coexist on the premise of P.

• AP: abandon consistency and pursue partition tolerance and availability. This is   Many choices in distributed system design  . Eureka cluster adopts the AP design idea.
• CP: abandon availability and pursue consistency and partition fault tolerance. zookeeper cluster.
• CA: give up partition tolerance, that is, do not partition, and do not consider the problem of network failure or node hanging, so as to achieve consistency and availability. Then the system will not be a standard distributed system, and our most commonly used relational data meets ca.

c) Summary

CAP is A proven theory: A distributed system can only meet two of the three items of Consistency, Availability and Partition tolerance at the same time. It can be used as our consideration standard for architecture design and technology selection. For most large-scale Internet application scenarios, there are many nodes, the deployment is scattered, and the current cluster scale is becoming larger and larger, so node failure and network failure are normal. In addition, to ensure that the service Availability reaches N 9 (99.99..%), and to achieve good response performance to improve the user experience, the following choices are generally made:   Ensure the strong Consistency of P and A, abandon C, and ensure the final Consistency  .

2. BASE theory

a) Strong consistency and final consistency

• Strong consistency: the consistency in CAP requires that the data of each node must be consistent at any time. It emphasizes strong consistency.
• Final consistency: allowed   The data of each node is inconsistent over a period of time  ， But after a period of time, the data of each node must be consistent. It emphasizes that   Consistency of final data  .

b) Concept

BASE is the abbreviation of basic available, soft state and finally consistent. BASE theory is an extension of AP in CAP. It obtains availability by sacrificing strong consistency. In case of failure, it is allowed to be partially unavailable, but to ensure the availability of core functions, it is allowed that the data is inconsistent for a period of time, but finally reach the consistent state. Transactions that satisfy the BASE theory are called“   Flexible transaction  ”.

• Basically available  : When the distributed system fails, it is allowed to lose some available functions,   Ensure core functions are available  . For example, if there is a problem with the transaction payment on the e-commerce website, the goods can still be browsed normally.
• Soft state  : Since strong consistency is not required, BASE allows the system to   There is an intermediate state   (also known as soft state), this state does not affect the system availability, such as the status of orders   "Payment in progress", "data synchronization in progress"   After the data is finally consistent, the status is changed to "successful".
• Final consistency  : Final agreement means   After a period of time, all node data will be consistent  . If the order is in the "paying" status, it will eventually change to "payment succeeded" or "payment failed", so that the order status can be agreed with the actual transaction result, but it will take a certain time to delay and wait.

3, Solution 2PC

1. What is 2PC

2PC is a two-stage commit protocol, which divides the whole transaction process into two stages: Prepare phase and commit phase. 2 refers to two stages, P refers to preparation phase and C refers to commit phase.

• Prepare phase:   The transaction manager sends a prepare message to each participant  ， Each database participant executes transactions locally and writes the local Undo/Redo log. At this time, the transaction is not committed. (Undo log records the data before modification, which is used for database rollback; Redo log records the modified data, which is used to write the data file after the transaction is committed)
• commit phase: if the transaction manager receives the execution failure or timeout message from the participant, it directly sends a rollback message to each participant; Otherwise, send a commit message; Participants perform commit or rollback operations according to the instructions of the transaction manager, and release the lock resources used in the transaction process. Note: Lock resources must be released at the final stage.

Success:

Failure:

2. XA of the solution

2PC   The traditional scheme is at the database level   For example, Oracle and MySQL support 2PC protocol. In order to unify standards and reduce unnecessary docking costs in the industry, it is necessary to formulate standardized processing models and interface standards. The international open standards organization Open Group defines the distributed transaction processing model DTP (Distributed Transaction Processing Reference Model).

The whole 2PC transaction process involves three roles: AP, RM and TM. AP refers to the use of 2PC distributed transactions   Application; RM means   Resource Manager  ， It controls branch transactions; TM means   Transaction manager  ， It controls the whole global transaction.

1) In   Preparation stage   RM performs actual business operations, but does not commit transactions, and resources are locked;

2) In   Submission phase   TM will accept the execution Reply of RM in the preparation phase. As long as any RM fails to execute, TM will notify all RM to perform rollback operation. Otherwise, TM will notify all RM to commit the transaction. The resource lock is released at the end of the commit phase.

Problems of XA scheme: 1. Local database is required to support XA protocol. 2. The resource lock needs to wait until the end of the two phases to be released, and the performance is poor.

3. Seata solution

a) Design idea of seata

One of Seata's design objectives is non-invasive to the business. Therefore, starting from the non-invasive 2PC scheme, Seata evolves on the basis of the traditional 2PC and solves the problems faced by the 2PC scheme.

Seata understands a distributed transaction as a global transaction that contains several branch transactions. The responsibility of a global transaction is to coordinate the branch transactions under its jurisdiction to reach an agreement, either successfully commit together or fail to roll back together. In addition, branch transactions are usually local transactions of a relational database. The following figure shows the relationship between global transactions and branch transactions:

Similar to the traditional 2PC model, Seata defines three components to protocol the processing process of distributed transactions:

• Transaction Coordinator (TC):   Transaction Coordinator  ， It is an independent middleware and requires   Independent deployment   It maintains the running state of global transactions, receives TM instructions, initiates the commit and rollback of global transactions, and is responsible for communicating with RM to coordinate the commit or rollback of each branch transaction.
• Transaction Manager (TM):   Transaction manager  ， TM needs to work embedded in the application, and it is responsible for   Start a global transaction  ， And finally initiate global commit or global rollback instructions to TC.
• Resource Manager (RM):   Control branch transactions  ， Be responsible for branch registration and status reporting, receive instructions from the transaction coordinator TC, and drive the submission and rollback of branch (local) transactions.

b) Seata's execution process

1. User service   TM requests TC to start a global transaction  ， Global transaction created successfully and   Generate a globally unique XID.
2. User service   RM registers branch transactions with TC  ， The branch transaction executes the new user logic in the user service and incorporates it into the   XID corresponds to the jurisdiction of global transactions  .
3. The user service performs branch transactions and inserts a record into the user table.
4. When the logic is executed to call the integration service remotely(   XID propagates in the context of the microservice invocation link  ). The RM of the integration service registers a branch transaction with the TC, which executes the logic of increasing the integration and brings it under the jurisdiction of the global transaction corresponding to XID.
5. The points service executes branch transactions, inserts a record into the points record table, and returns to the user service after execution.
6. User service branch transaction execution completed.
7. TM initiates a global commit or rollback resolution for XID to TC  .
8. TC schedules all branch transactions under XID   Complete commit or rollback request  .

c) Implementation of Seata

See Spring Cloud Alibaba Seata for details

4. Seata and traditional 2PC

• In terms of architecture level,   Traditional 2PC   The rm of the scheme is actually   Database layer  ， RM is essentially the database itself, which is implemented through XA protocol, and   Seata's RM is deployed as a middleware layer in the form of jar packages   On the application side.
• In terms of two-stage submission, the traditional 2PC determines whether the decision in the second stage is commit or rollback,   The locks of transactional resources are not released until Phase2 is completed  . and   Seata's approach is to commit local transactions in phase 1  ， In this way, the lock holding time of Phase2 can be saved, and the overall   increase of efficiency  .

4, TCC of solution

1. What is TCC

TCC is the abbreviation of try, confirm and cancel. TCC requires each branch transaction to implement three operations: preprocess try, confirm and cancel   Conduct business inspection and resource reservation  ，  Confirm business confirmation  ，  Cancel implements an operation opposite to try, that is, rollback  . TM first initiates the try operation of all branch transactions. If the try operation of any branch transaction fails, TM will initiate the cancel operation of all branch transactions. If all the try operations are successful, TM will initiate the confirm operation of all branch transactions. If the Confirm/Cancel operation fails, TM will retry.

Success:

Failure:

TCC is divided into three stages:

• Try stage is to do   Business check (consistency) and resource reservation (isolation)  ， This stage is only a preliminary operation, and it can really form a complete business logic together with the subsequent Confirm.
• The confirm phase is to do   Confirm submission  ， In the Try phase, confirm is executed after all branch transactions are executed successfully. Generally, if TCC is adopted, it is considered that there will be no error in the confirm phase. That is:   As long as Try succeeds, confirm will succeed  . If an error occurs in the confirm phase, a retry mechanism or manual processing shall be introduced.
• The Cancel phase is a business execution error   Business cancellation of branch transactions in the state of rollback  ， reserve   Resource release. Generally, TCC is used   I think the Cancel stage will also be successful   of If an error occurs in the Cancel phase, a retry mechanism or manual processing shall be introduced.

2. TCC solution

Frame name	Github address
tcc-transaction	https://github.com/changmingxie/tcc-transaction
Hmily	https://github.com/yu199195/hmily
ByteTCC	https://github.com/liuyangming/ByteTCC
EasyTransaction	https://github.com/QNJR-GROUP/EasyTransaction

3. Problems needing attention of TCC

a) Empty rollback

When the TCC resource Try method is not called, the two-stage Cancel method is called. The Cancel method needs to   It is recognized that this is an empty rollback  ， Then it returns success directly.

Cause: when a branch transaction is in service downtime or network abnormality, the branch transaction call is recorded as failure. In fact, the Try phase is not executed at this time. After fault recovery, the distributed transaction rollback will call the Cancel method of the second phase, resulting in an empty rollback.

Solution: identify the empty rollback. You need to know whether a phase is executed. If it is executed, it is normal rollback; If it is not executed, it is an empty rollback. As mentioned earlier, TM generates a global transaction record when initiating a global transaction, and the global transaction ID runs through the whole distributed transaction call chain. An additional branch transaction record table is added, including global transaction ID and branch transaction ID. in the first stage, a record will be inserted in the Try method, indicating that the first stage has been executed.

//In cancel, cancel the empty rollback process. If the try is not executed, cancel is not allowed
if(accountInfoDao.isExistTry(transId)<=0){
    log.info("bank1 Empty rollback processing, try Not implemented, not allowed cancel implement,xid:{}",transId);
    return ;
}

b) Idempotent

In order to ensure that the two-phase commit retry mechanism of TCC will not cause data inconsistency, the two-phase Try, Confirm and Cancel interfaces of TCC are required to ensure idempotence, so that resources will not be reused or released. If idempotent control is not done well, it is likely to lead to serious problems such as data inconsistency.

//Currently, idempotent judgment is performed in try. Local_ try_ Is there a try log record in the log table? If so, it will not be executed again
if(accountInfoDao.isExistTry(transId)>0){
    log.info("bank1 try Already executed, no need to repeat,xid:{}",transId);
    return ;
}

c) Hang

Suspension means that for a distributed transaction, the two-stage Cancel interface executes before the Try interface.

Cause: when RPC calls the branch transaction Try, first register the branch transaction and then execute the RPC call. If the network of RPC calls is congested at this time, the RPC call usually has a timeout time,   RPC timeout   TM will notify RM later   RollBACK   The distributed transaction may be rolled back before the RPC request reaches the participant for real execution, and a Try method reserves business resources.

Solution: if the implementation of phase II is completed, the implementation of that phase can no longer be continued. When executing a phase transaction, it is judged that under the global transaction,   Does the branch transaction record table already have a two-phase transaction record  ， If so, Try is not performed.

//Try suspension processing. If one of cancel and confirm has been executed, try will not be executed again
if(accountInfoDao.isExistConfirm(transId)>0 || accountInfoDao.isExistCancel(transId)>0){
    log.info("bank1 try Suspension treatment  cancel or confirm Executed, not allowed try,xid:{}",transId);
    return ;
}

4，Hmily

Project source code: cloud DTX TCC

a) Import database

The download address of sql file is DTX TCC sql

b) Engineering configuration

All projects involving distributed transactions require different configurations

maven configuration

<!-- hmily rely on -->
<dependency> 
    <groupId>org.dromara</groupId>
    <artifactId>hmily‐springcloud</artifactId>
    <version>2.0.4‐RELEASE</version>
</dependency>

Add hmily to application.yaml

org:
  dromara:
    hmily:
      serializer: kryo
      recoverDelayTime: 30
      retryMax: 30
      scheduledDelay: 30
      scheduledThreadMax: 10
      repositorySupport: db
      #For the initiator, set this property to true. Party is false.
      started: true
      hmilyDbConfig:
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://localhost:3306/hmily?useUnicode=true
        username: root
        password: 123456

Inject hmily's configuration Bean

@Bean
public HmilyTransactionBootstrap hmilyTransactionBootstrap(HmilyInitService hmilyInitService){
    HmilyTransactionBootstrap hmilyTransactionBootstrap = new HmilyTransactionBootstrap(hmilyInitService);
    hmilyTransactionBootstrap.setSerializer(env.getProperty("org.dromara.hmily.serializer"));
    hmilyTransactionBootstrap.setRecoverDelayTime(Integer.parseInt(env.getProperty("org.dromara.hmily.recoverDelayTime")));
    hmilyTransactionBootstrap.setRetryMax(Integer.parseInt(env.getProperty("org.dromara.hmily.retryMax")));
    hmilyTransactionBootstrap.setScheduledDelay(Integer.parseInt(env.getProperty("org.dromara.hmily.scheduledDelay")));
    hmilyTransactionBootstrap.setScheduledThreadMax(Integer.parseInt(env.getProperty("org.dromara.hmily.scheduledThreadMax")));
    hmilyTransactionBootstrap.setRepositorySupport(env.getProperty("org.dromara.hmily.repositorySupport"));
    hmilyTransactionBootstrap.setStarted(Boolean.parseBoolean(env.getProperty("org.dromara.hmily.started")));
    HmilyDbConfig hmilyDbConfig = new HmilyDbConfig();
    hmilyDbConfig.setDriverClassName(env.getProperty("org.dromara.hmily.hmilyDbConfig.driverClassName"));
    hmilyDbConfig.setUrl(env.getProperty("org.dromara.hmily.hmilyDbConfig.url"));
    hmilyDbConfig.setUsername(env.getProperty("org.dromara.hmily.hmilyDbConfig.username"));
    hmilyDbConfig.setPassword(env.getProperty("org.dromara.hmily.hmilyDbConfig.password"));
    hmilyTransactionBootstrap.setHmilyDbConfig(hmilyDbConfig);
    return hmilyTransactionBootstrap;
}

Add annotation on startup class

@ComponentScan({"org.dromara.hmily"})

c) Caller (bank1) implementation

Code implementation: AccountInfoServiceImpl

try:  
    try Idempotent check
    try Suspension treatment
    Check that the balance is sufficient to deduct the amount
    Deduction amount
confirm: 
    empty
cancel
    cancel Idempotent check
    cancel Null rollback processing
    Increase available balance

Note: when calling bank2 remotely, annotate @ Hmily on the interface called by feign

d) Participant (bank2) implementation

Code implementation: AccountInfoServiceImpl

try: 
    empty
confirm: 
    confirm Idempotent check
    Formal increase amount
cancel: 
    empty

e) Summary

If the TCC transaction processing flow is compared with 2PC two-phase commit,   2PC is usually at the cross database DB level  ， and   TCC is handled at the application level  ， It needs to be implemented through business logic. The implementation of this distributed transaction   advantage   The is that the application can define the granularity of data operation by itself, so that   Reduce lock conflicts and improve throughput   It's possible.

and   deficiencies   It lies in the application   Very invasive  ， Each branch of business logic needs to implement three operations: try, confirm and cancel. In addition, its   It is also difficult to realize  ， Different rollback strategies need to be implemented according to different failure causes such as network status and system failure.

5, Reliable message final consistency of solution

Project source code: cloud DTX txmsg

1. What is the final consistency of reliable messages

The reliable message final consistency scheme refers to when   The transaction initiator executes the local transaction and issues a message  ， Transaction participants (message consumers) must be able to receive messages and process transactions successfully. This scheme emphasizes that as long as messages are sent to transaction participants, the final transaction must be consistent.

Problems to be solved for reliable messages:

• Atomicity of local transactions and message sending
• //First message if the database operation is wrong, the message has been sent   begin   transaction； // 1. Send MQ //2. Database operation   commit   transation; // If the database times out, the database rolls back, but the message may have been sent   begin   transaction； // 1. Database operation / / 2. Sending MQ   commit   transation;
• Reliability of message accepted by transaction participants
• Transaction participants must be able to receive messages from the message queue. If receiving messages fails, they can receive messages repeatedly.
• The problem of repeated consumption of messages
• Due to the existence of network 2, if a consumption node times out but the consumption is successful, the message middleware will deliver the message repeatedly, resulting in repeated consumption of the message. To solve the problem of repeated consumption of messages, it is necessary to realize the method idempotency of transaction participants.

2. RocketMQ transaction message scheme

• Producer   Send transaction message  : The producer (MQ sender) sends a transaction message to the MQ Server, which marks the message status as Prepared(   Ready state  )， Note that the message consumer (MQ subscriber) is   Unable to consume   Yes.
• MQ Server   Response message   Sending succeeded: when MQ server receives the message sent to by Producer, it responds that sending succeeded, indicating that MQ has received the message.
• Producer execution   Local transaction  : The producer side executes business code logic through   Local database transaction control  .
• Message delivery  : If Producer local transaction   Successful execution   Automatically sent to MQServer   commit   Message. At this time, the MQ subscriber (point service) is the normal consumption message; If Producer local transaction   Execution failed   Automatically sent to MQServer   rollback   After receiving the rollback message, the MQ Server will delete the "add points message". The MQ subscriber (points service) consumes the message,   If the consumption is successful, respond ack to MQ  ， Otherwise, the message will be received repeatedly. Here ack will respond automatically by default, that is, when the program is running normally, ACK will be responded automatically.
• Transaction backcheck  : If the Producer side is executed   During a local transaction, the execution end hangs or times out  ， MQ Server will keep asking   Other producers in the same group to get the transaction execution status  ， This process is called transaction backcheck. MQ Server will decide whether to post messages according to the transaction backcheck results.

3. RocketMQ implements the final consistency transaction of reliable messages

a)SQL

bank1

CREATE DATABASE `bank1` CHARACTER
SET 'utf8' COLLATE 'utf8_general_ci';

DROP TABLE
IF EXISTS `account_info`;

CREATE TABLE `account_info` (
    `id` BIGINT (20) NOT NULL AUTO_INCREMENT,
    `account_name` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Name of head of household',
 `account_no` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Bank card No',
 `account_password` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Account password',
 `account_balance` DOUBLE NULL DEFAULT NULL COMMENT 'account balance',
 PRIMARY KEY (`id`) USING BTREE
) ENGINE = INNODB AUTO_INCREMENT = 5 CHARACTER
SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic;

INSERT INTO `account_info`
VALUES
    (
        2,
        'Zhang San's account',
        '1',
        '',
        10000
    );

DROP TABLE
IF EXISTS `de_duplication`;

CREATE TABLE `de_duplication` (
    `tx_no` VARCHAR (64) COLLATE utf8_bin NOT NULL,
    `create_time` datetime (0) NULL DEFAULT NULL,
    PRIMARY KEY (`tx_no`) USING BTREE
) ENGINE = INNODB CHARACTER
SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic;

View Code

bank2

CREATE DATABASE `bank2` CHARACTER
SET 'utf8' COLLATE 'utf8_general_ci';

DROP TABLE
IF EXISTS `account_info`;

CREATE TABLE `account_info` (
    `id` BIGINT (20) NOT NULL AUTO_INCREMENT,
    `account_name` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Name of head of household',
 `account_no` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Bank card No',
 `account_password` VARCHAR (100) CHARACTER
SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Account password',
 `account_balance` DOUBLE NULL DEFAULT NULL COMMENT 'account balance',
 PRIMARY KEY (`id`) USING BTREE
) ENGINE = INNODB AUTO_INCREMENT = 5 CHARACTER
SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic;

INSERT INTO `account_info`
VALUES
    (
        3,
        'Li Si's account',
        '2',
        NULL,
        0
    );

CREATE TABLE `de_duplication` (
    `tx_no` VARCHAR (64) COLLATE utf8_bin NOT NULL,
    `create_time` datetime (0) NULL DEFAULT NULL,
    PRIMARY KEY (`tx_no`) USING BTREE
) ENGINE = INNODB CHARACTER
SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic;

View Code

b) Installing RocketMQ

c) Engineering configuration

maven

<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-spring-boot-starter</artifactId>
    <version>2.0.2</version>
</dependency>

properties configuration

rocketmq.producer.group = producer_bank2
rocketmq.name‐server = 127.0.0.1:9876

d)bank1

Service: AccountInfoServiceImpl

//Two methods
//1. Send a transfer message to mq
//2. Update the account and deduct the amount (ensure idempotency through transaction id)

Controller: AccountInfoController

//Generate the transaction id and call the messaging interface of the service

message: ProducerTxmsgListener

//There are two methods, executelocetransaction and checkLocalTransaction
//The callback method after the transaction message is sent. At this time, ensure the local transaction, call the Service to deduct the amount, and change the message to COMMIT (consumable state). If an exception is caught, change the message to ROLLBACK
//Transaction backcheck. Query whether the caller has processed it. If it has been processed, the message needs to be modified to COMMIT consumable. Otherwise, it is in the UNKOWN state.

e)bank2

Service: AccountInfoServiceImpl

//Update account bank2 and increase the amount. (ensure idempotency through transaction id)

message: TxmsgConsumer

//Listen to the message topic sent by bank1 and call Service to increase the amount

4. Summary

The ultimate consistency of reliable messages is   Ensure that messages are delivered from the producer to the consumer through message middleware   In this case, RocketMQ is used as the message middleware. RocketMQ mainly solves two functions:

• Atomicity of local transactions and message sending.
• Reliability of messages received by transaction participants.

Reliable message final consistency transaction fit   Scenarios with long execution cycle and low real-time requirements  . After the introduction of message mechanism, synchronous transaction operations become message based   asynchronous   The operation avoids the influence of synchronous blocking operation in distributed transactions, and realizes the synchronization of two services   decoupling  .

6, Best effort notification of Solutions

Source code: cloud DTX notify

1. What is best effort notification

The initiating party notifies through a certain mechanism   Make every effort to inform the receiver of the business processing results  .

• There is a certain   Message repetition notification mechanism  . Because the party receiving the notification may not receive the notification, there must be a certain mechanism to repeat the notification for the message.
• Message proofreading mechanism  . If the receiver has not been notified as far as possible, or the receiver wants to consume the message again after consuming it, you can   The receiving party shall actively query the notifying party for messages   Information to meet needs.

2. Similarities and differences between best effort notification and reliable message consistency

• Different ideas: reliable message consistency, initiation   The notifying party needs to ensure that the message is sent  ， And send the message to the receiving notifier. The reliability of the message is guaranteed by the initiating notifier. The initiating notifier tries its best to notify the receiving notifier of the business processing result, but the message may not be received. At this time, the receiving notifier needs to actively call the interface of the initiating notifier to query the business processing result,   The key to the reliability of the notice lies in the party receiving the notice  .
• Different business scenarios: reliable message consistency focuses on   Transaction consistency in the transaction process  ， Complete the transaction asynchronously. Best effort notification focuses on   Post transaction notification transaction  ， Inform the trading results reliably.
• Different technical solutions: reliable message consistency needs to be solved   Message consistency from sending to receiving  ， That is, the message is sent and received; the best effort notification cannot guarantee the consistency of the message from sending to receiving, but only provides the reliability mechanism of message receiving. The reliability mechanism is,   Try your best to notify the receiver of the message  ， When the message cannot be received by the receiver, the   The receiver actively queries the message   (business processing results).

3. Solution

a) Solution 1:

Specific process:

• The originating notifier sends the notification to MQ. The notification is sent to MQ using the normal message mechanism.
• The receiving party listens to MQ.
• The receiving notifier receives the message, and the service processing is completed to respond to the ack.
• If the party receiving the notice   If there is no response to ack, MQ will repeat the notification  . MQ will gradually increase the notification interval according to the interval of 1min, 5min, 10min, 30min, 1h, 2h, 5h and 10h (if MQ adopts rocketMq, it can be configured in the broker) until the upper limit of the time window required by the notification is reached.
• The receiving party can check the consistency of the message through the message checking interface.

b) Solution 2:

Unlike option 1   The application sends notifications to the recipient  ， As shown below:

Specific process:

• The initiating notifier sends the notification to MQ: the transaction message guarantee in the reliable message consistency scheme is used   Local transactions and atomicity of messages  ， Finally, the notification will be sent to MQ first.
• The notifier listens to MQ and receives messages from MQ. If the notifier does not respond to ack, MQ will repeat the notification.
• Notification procedure   Through Internet interface protocol (such as http, webservice)   Call the receive notification scheme interface  ， The notification is completed. If the notifier calls the notification scheme interface successfully, the notification is successful, that is, the consumption of MQ messages is successful, and MQ will no longer deliver notification messages to the notifier.
• The receiving party can check the consistency of the message through the message checking interface.

c) Comparison of two schemes

• In scheme 1, the receiving notifier interfaces with MQ, that is, the receiving notification scheme listens to MQ. This scheme mainly   Notification between application and internal application  .
• In scheme 2, the notifier interfaces with MQ. The notifier listens to MQ. After receiving MQ messages, the notifier calls the receiving notifier through the Internet interface protocol. This scheme is mainly applied to   Notification between external applications  ， For example, Alipay and WeChat pay the result notice.

4. Best effort notification

a)sql

bank1_pay.sql

CREATE DATABASE /*!32312 IF NOT EXISTS*/`bank1_pay` /*!40100 DEFAULT CHARACTER SET utf8 */;

USE `bank1_pay`;

/*Table structure for table `account_pay` */

DROP TABLE IF EXISTS `account_pay`;

CREATE TABLE `account_pay` (
  `id` varchar(64) COLLATE utf8_bin NOT NULL,
  `account_no` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'account number',
  `pay_amount` double DEFAULT NULL COMMENT 'Recharge balance',
  `result` varchar(20) COLLATE utf8_bin DEFAULT NULL COMMENT 'Recharge result:success，fail',
  PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC;

/*Data for the table `account_pay` */

insert  into `account_pay`(`id`,`account_no`,`pay_amount`,`result`) values ('5678ef0a-1ff0-4cfd-97ac-640d749d596f','1',2,'success'),('7d7d469c-f100-4066-b927-014c0c3aa010','1',2,'success'),('947fafad-c19c-46bc-b0f0-43703a124fd4','1',2,'success');

View Code

bank1.sql

CREATE DATABASE /*!32312 IF NOT EXISTS*/`bank1` /*!40100 DEFAULT CHARACTER SET utf8 */;

USE `bank1`;

/*Table structure for table `account_info` */

DROP TABLE IF EXISTS `account_info`;

CREATE TABLE `account_info` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `account_name` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Name of head of household',
  `account_no` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Bank card No',
  `account_password` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Account password',
  `account_balance` double DEFAULT NULL COMMENT 'account balance',
  PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC;

/*Data for the table `account_info` */

insert  into `account_info`(`id`,`account_name`,`account_no`,`account_password`,`account_balance`) values (2,'Zhang San','1',NULL,1000);

/*Table structure for table `de_duplication` */

DROP TABLE IF EXISTS `de_duplication`;

CREATE TABLE `de_duplication` (
  `tx_no` varchar(64) COLLATE utf8_bin NOT NULL,
  `create_time` datetime DEFAULT NULL,
  PRIMARY KEY (`tx_no`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC;

View Code

b) Engineering configuration

Consistency between basic configuration and reliable message

c) pay payer

Service: AccountPayServiceImpl

//Two methods
//1. Insert the recharge record. Generate the transaction id and send the transaction id and recharge information to the MQ queue
//2. Query recharge records. Provides a query to the caller.

Controller: AccountPayController

//Directly call the method in the Service to insert the recharge record

d)bank1

Service: AccountInfoServiceImpl

//Two methods
//1. Update the account amount. The idempotency of the update is guaranteed according to the transaction id.
//2. Remotely call pay to query the recharge result. If the status changes, update the current account at the same time.

message: NotifyMsgListener

//Listen for messages. Update account amount by calling Service, idempotent update.

Controller: AccountInfoController

//Query recharge result of calling Service

5. Summary

The best effort notification scheme is in distributed transactions   The one with the lowest requirements for consistency  , It is applicable to some businesses with low final consistency and time sensitivity; The best effort notification scheme needs to realize the following functions:

• Message repetition notification mechanism.
• Message proofreading mechanism. Actively call the interface to query and modify.

7, Comparison of four distributed transactions

2PC:   The biggest criticism is one   Blocking protocol  . RM needs to wait for TM's decision after executing branch transactions. At this time, the service will block and lock resources. Due to its blocking mechanism and high worst-case time complexity, this design can not meet the needs of expansion with the increase of the number of services involved in transactions, and it is difficult to be used in distributed services with high concurrency and long-running transactions.

TCC  : If the TCC transaction processing flow is compared with 2pc two-phase commit,   2PC   Usually in   Cross database DB level, and   TCC is handled at the application level  ， It needs to be implemented through business logic. The advantage of this distributed transaction implementation is that it allows applications to define the granularity of data operations   Reduce lock conflicts and improve throughput   It's possible. The disadvantage is that it is very intrusive to applications. Each branch of business logic needs to implement three operations: try, confirm and cancel. In addition, its   It is also difficult to realize  ， Different rollback strategies need to be implemented according to different failure causes such as network status and system failure. Typical usage scenarios: login, coupon, etc.

Reliable message final consistency transaction:   fit   Scenarios with long execution cycle and low real-time requirements  . After the introduction of message mechanism, the synchronous transaction operation becomes asynchronous operation based on message execution, which avoids the influence of synchronous blocking operation in distributed transactions, and realizes the decoupling of the two services. Typical usage scenarios:   Register to send points and login to send coupons   Wait.

Best effort notification:   It is one of the lowest requirements in distributed transactions, and is suitable for some services with low final consistency and time sensitivity; The initiating party is allowed to handle the business failure and actively handle the failure after the receiving party receives the notice. No matter how the initiating party handles the result, it will not affect the subsequent processing of the receiving party; The initiating notifier shall provide the query execution interface for   Receiving party's proofreading results  . Typical usage scenarios:   Bank notice and payment result   Notice, etc.

Original link:
http://www.cnblogs.com/bbgs-xc/p/14456917.html

If you think this article is helpful to you, pay attention to and support it!