1, Concept of distributed transaction
1. What is a transaction
A transaction can be regarded as a big activity. It consists of different small activities. These activities either succeed or fail.
2. Local affairs
Four characteristics of database transactions ACID:
- A(Atomic): Atomicity , All operations constituting a transaction are either completed or not executed at all. It is impossible to have partial success and partial failure.
- C(Consistency): uniformity , Before and after the transaction, the consistency constraint of the database is not broken. For example, if Zhang San transfers 100 yuan to Li Si, the data before and after the transfer is correct, which is called consistency. If Zhang San transfers 100 yuan out and Li Si's account does not increase 100 yuan, there is a data error and consistency is not achieved.
- I(Isolation): Isolation , Transactions in the database are generally concurrent. Isolation means that the execution of two concurrent transactions does not interfere with each other. One transaction cannot see the intermediate state of the running process of other transactions. By configuring the transaction isolation level, problems such as dirty reads and duplicate reads can be avoided.
- D(Durability): persistence , After the transaction is completed, the data changes made by the transaction will be persisted to the database and will not be rolled back.
When implementing a database transaction, all operations involved in a transaction will be incorporated into an inseparable execution unit. All operations in the execution unit will either succeed or fail. As long as any operation fails, the whole transaction will roll back.
3. Distributed transaction
A distributed system will split an application system into multiple services that can be deployed independently, so it requires remote cooperation between services to complete transaction operations. In this distributed system environment, transactions completed by remote cooperation between different services through the network are called distributed transactions, such as user registration and delivery of points, creation of orders and inventory reduction, Bank transfer transactions are distributed transactions.
4. Scenario of distributed transaction generation
- A typical scenario is the microservice architecture, where microservices pass through each other Remote call complete transaction Operation. For example, order microservices and inventory microservices request inventory microservices to reduce inventory while placing an order. in short: Generate distributed transactions across JVM processes .
- A single system accesses multiple database instances, Generate distributed transactions across database instances .
- Multiple services access the same database instance , For example, order microservices and inventory microservices will generate distributed transactions even if they access the same database Cross JVM processes , Two microservices hold different database links for database operations, and distributed transactions are generated at this time.
2, Basic theory of distributed transaction
1. CAP theory
a) Concept
CAP is the abbreviation of Consistency, Availability and Partition tolerance, which means Consistency, Availability and Partition tolerance respectively.
b) Combination mode
In all distributed transaction scenarios, the three characteristics of CAP will not be available at the same time, because C and A cannot coexist on the premise of P.
- AP: abandon consistency and pursue partition tolerance and availability. This is Many choices in distributed system design . Eureka cluster adopts the AP design idea.
- CP: abandon availability and pursue consistency and partition fault tolerance. zookeeper cluster.
- CA: give up partition tolerance, that is, do not partition, and do not consider the problem of network failure or node hanging, so as to achieve consistency and availability. Then the system will not be a standard distributed system, and our most commonly used relational data meets ca.
c) Summary
CAP is A proven theory: A distributed system can only meet two of the three items of Consistency, Availability and Partition tolerance at the same time. It can be used as our consideration standard for architecture design and technology selection. For most large-scale Internet application scenarios, there are many nodes, the deployment is scattered, and the current cluster scale is becoming larger and larger, so node failure and network failure are normal. In addition, to ensure that the service Availability reaches N 9 (99.99..%), and to achieve good response performance to improve the user experience, the following choices are generally made: Ensure the strong Consistency of P and A, abandon C, and ensure the final Consistency .
2. BASE theory
a) Strong consistency and final consistency
- Strong consistency: the consistency in CAP requires that the data of each node must be consistent at any time. It emphasizes strong consistency.
- Final consistency: allowed The data of each node is inconsistent over a period of time , But after a period of time, the data of each node must be consistent. It emphasizes that Consistency of final data .
b) Concept
BASE is the abbreviation of basic available, soft state and finally consistent. BASE theory is an extension of AP in CAP. It obtains availability by sacrificing strong consistency. In case of failure, it is allowed to be partially unavailable, but to ensure the availability of core functions, it is allowed that the data is inconsistent for a period of time, but finally reach the consistent state. Transactions that satisfy the BASE theory are called“ Flexible transaction ”.
- Basically available : When the distributed system fails, it is allowed to lose some available functions, Ensure core functions are available . For example, if there is a problem with the transaction payment on the e-commerce website, the goods can still be browsed normally.
- Soft state : Since strong consistency is not required, BASE allows the system to There is an intermediate state (also known as soft state), this state does not affect the system availability, such as the status of orders "Payment in progress", "data synchronization in progress" After the data is finally consistent, the status is changed to "successful".
- Final consistency : Final agreement means After a period of time, all node data will be consistent . If the order is in the "paying" status, it will eventually change to "payment succeeded" or "payment failed", so that the order status can be agreed with the actual transaction result, but it will take a certain time to delay and wait.
3, Solution 2PC
1. What is 2PC
2PC is a two-stage commit protocol, which divides the whole transaction process into two stages: Prepare phase and commit phase. 2 refers to two stages, P refers to preparation phase and C refers to commit phase.
- Prepare phase: The transaction manager sends a prepare message to each participant , Each database participant executes transactions locally and writes the local Undo/Redo log. At this time, the transaction is not committed. (Undo log records the data before modification, which is used for database rollback; Redo log records the modified data, which is used to write the data file after the transaction is committed)
- commit phase: if the transaction manager receives the execution failure or timeout message from the participant, it directly sends a rollback message to each participant; Otherwise, send a commit message; Participants perform commit or rollback operations according to the instructions of the transaction manager, and release the lock resources used in the transaction process. Note: Lock resources must be released at the final stage.
Success:
Failure:
2. XA of the solution
2PC The traditional scheme is at the database level For example, Oracle and MySQL support 2PC protocol. In order to unify standards and reduce unnecessary docking costs in the industry, it is necessary to formulate standardized processing models and interface standards. The international open standards organization Open Group defines the distributed transaction processing model DTP (Distributed Transaction Processing Reference Model).
The whole 2PC transaction process involves three roles: AP, RM and TM. AP refers to the use of 2PC distributed transactions Application; RM means Resource Manager , It controls branch transactions; TM means Transaction manager , It controls the whole global transaction.
1) In Preparation stage RM performs actual business operations, but does not commit transactions, and resources are locked;
2) In Submission phase TM will accept the execution Reply of RM in the preparation phase. As long as any RM fails to execute, TM will notify all RM to perform rollback operation. Otherwise, TM will notify all RM to commit the transaction. The resource lock is released at the end of the commit phase.
Problems of XA scheme: 1. Local database is required to support XA protocol. 2. The resource lock needs to wait until the end of the two phases to be released, and the performance is poor.
3. Seata solution
a) Design idea of seata
One of Seata's design objectives is non-invasive to the business. Therefore, starting from the non-invasive 2PC scheme, Seata evolves on the basis of the traditional 2PC and solves the problems faced by the 2PC scheme.
Seata understands a distributed transaction as a global transaction that contains several branch transactions. The responsibility of a global transaction is to coordinate the branch transactions under its jurisdiction to reach an agreement, either successfully commit together or fail to roll back together. In addition, branch transactions are usually local transactions of a relational database. The following figure shows the relationship between global transactions and branch transactions:
Similar to the traditional 2PC model, Seata defines three components to protocol the processing process of distributed transactions:
- Transaction Coordinator (TC): Transaction Coordinator , It is an independent middleware and requires Independent deployment It maintains the running state of global transactions, receives TM instructions, initiates the commit and rollback of global transactions, and is responsible for communicating with RM to coordinate the commit or rollback of each branch transaction.
- Transaction Manager (TM): Transaction manager , TM needs to work embedded in the application, and it is responsible for Start a global transaction , And finally initiate global commit or global rollback instructions to TC.
- Resource Manager (RM): Control branch transactions , Be responsible for branch registration and status reporting, receive instructions from the transaction coordinator TC, and drive the submission and rollback of branch (local) transactions.
b) Seata's execution process
- User service TM requests TC to start a global transaction , Global transaction created successfully and Generate a globally unique XID.
- User service RM registers branch transactions with TC , The branch transaction executes the new user logic in the user service and incorporates it into the XID corresponds to the jurisdiction of global transactions .
- The user service performs branch transactions and inserts a record into the user table.
- When the logic is executed to call the integration service remotely( XID propagates in the context of the microservice invocation link ). The RM of the integration service registers a branch transaction with the TC, which executes the logic of increasing the integration and brings it under the jurisdiction of the global transaction corresponding to XID.
- The points service executes branch transactions, inserts a record into the points record table, and returns to the user service after execution.
- User service branch transaction execution completed.
- TM initiates a global commit or rollback resolution for XID to TC .
- TC schedules all branch transactions under XID Complete commit or rollback request .
c) Implementation of Seata
See Spring Cloud Alibaba Seata for details
4. Seata and traditional 2PC
- In terms of architecture level, Traditional 2PC The rm of the scheme is actually Database layer , RM is essentially the database itself, which is implemented through XA protocol, and Seata's RM is deployed as a middleware layer in the form of jar packages On the application side.
- In terms of two-stage submission, the traditional 2PC determines whether the decision in the second stage is commit or rollback, The locks of transactional resources are not released until Phase2 is completed . and Seata's approach is to commit local transactions in phase 1 , In this way, the lock holding time of Phase2 can be saved, and the overall increase of efficiency .
4, TCC of solution
1. What is TCC
TCC is the abbreviation of try, confirm and cancel. TCC requires each branch transaction to implement three operations: preprocess try, confirm and cancel Conduct business inspection and resource reservation , Confirm business confirmation , Cancel implements an operation opposite to try, that is, rollback . TM first initiates the try operation of all branch transactions. If the try operation of any branch transaction fails, TM will initiate the cancel operation of all branch transactions. If all the try operations are successful, TM will initiate the confirm operation of all branch transactions. If the Confirm/Cancel operation fails, TM will retry.
Success:
Failure:
TCC is divided into three stages:
- Try stage is to do Business check (consistency) and resource reservation (isolation) , This stage is only a preliminary operation, and it can really form a complete business logic together with the subsequent Confirm.
- The confirm phase is to do Confirm submission , In the Try phase, confirm is executed after all branch transactions are executed successfully. Generally, if TCC is adopted, it is considered that there will be no error in the confirm phase. That is: As long as Try succeeds, confirm will succeed . If an error occurs in the confirm phase, a retry mechanism or manual processing shall be introduced.
- The Cancel phase is a business execution error Business cancellation of branch transactions in the state of rollback , reserve Resource release. Generally, TCC is used I think the Cancel stage will also be successful of If an error occurs in the Cancel phase, a retry mechanism or manual processing shall be introduced.
2. TCC solution
Frame name | Github address |
tcc-transaction | https://github.com/changmingxie/tcc-transaction |
Hmily | https://github.com/yu199195/hmily |
ByteTCC | https://github.com/liuyangming/ByteTCC |
EasyTransaction | https://github.com/QNJR-GROUP/EasyTransaction |
3. Problems needing attention of TCC
a) Empty rollback
When the TCC resource Try method is not called, the two-stage Cancel method is called. The Cancel method needs to It is recognized that this is an empty rollback , Then it returns success directly.
Cause: when a branch transaction is in service downtime or network abnormality, the branch transaction call is recorded as failure. In fact, the Try phase is not executed at this time. After fault recovery, the distributed transaction rollback will call the Cancel method of the second phase, resulting in an empty rollback.
Solution: identify the empty rollback. You need to know whether a phase is executed. If it is executed, it is normal rollback; If it is not executed, it is an empty rollback. As mentioned earlier, TM generates a global transaction record when initiating a global transaction, and the global transaction ID runs through the whole distributed transaction call chain. An additional branch transaction record table is added, including global transaction ID and branch transaction ID. in the first stage, a record will be inserted in the Try method, indicating that the first stage has been executed.
//In cancel, cancel the empty rollback process. If the try is not executed, cancel is not allowed if(accountInfoDao.isExistTry(transId)<=0){ log.info("bank1 Empty rollback processing, try Not implemented, not allowed cancel implement,xid:{}",transId); return ; }
b) Idempotent
In order to ensure that the two-phase commit retry mechanism of TCC will not cause data inconsistency, the two-phase Try, Confirm and Cancel interfaces of TCC are required to ensure idempotence, so that resources will not be reused or released. If idempotent control is not done well, it is likely to lead to serious problems such as data inconsistency.
//Currently, idempotent judgment is performed in try. Local_ try_ Is there a try log record in the log table? If so, it will not be executed again if(accountInfoDao.isExistTry(transId)>0){ log.info("bank1 try Already executed, no need to repeat,xid:{}",transId); return ; }
c) Hang
Suspension means that for a distributed transaction, the two-stage Cancel interface executes before the Try interface.
Cause: when RPC calls the branch transaction Try, first register the branch transaction and then execute the RPC call. If the network of RPC calls is congested at this time, the RPC call usually has a timeout time, RPC timeout TM will notify RM later RollBACK The distributed transaction may be rolled back before the RPC request reaches the participant for real execution, and a Try method reserves business resources.
Solution: if the implementation of phase II is completed, the implementation of that phase can no longer be continued. When executing a phase transaction, it is judged that under the global transaction, Does the branch transaction record table already have a two-phase transaction record , If so, Try is not performed.
//Try suspension processing. If one of cancel and confirm has been executed, try will not be executed again if(accountInfoDao.isExistConfirm(transId)>0 || accountInfoDao.isExistCancel(transId)>0){ log.info("bank1 try Suspension treatment cancel or confirm Executed, not allowed try,xid:{}",transId); return ; }
4,Hmily
Project source code: cloud DTX TCC
a) Import database
The download address of sql file is DTX TCC sql
b) Engineering configuration
All projects involving distributed transactions require different configurations
maven configuration
<!-- hmily rely on --> <dependency> <groupId>org.dromara</groupId> <artifactId>hmily‐springcloud</artifactId> <version>2.0.4‐RELEASE</version> </dependency>
Add hmily to application.yaml
org: dromara: hmily: serializer: kryo recoverDelayTime: 30 retryMax: 30 scheduledDelay: 30 scheduledThreadMax: 10 repositorySupport: db #For the initiator, set this property to true. Party is false. started: true hmilyDbConfig: driverClassName: com.mysql.jdbc.Driver url: jdbc:mysql://localhost:3306/hmily?useUnicode=true username: root password: 123456
Inject hmily's configuration Bean
@Bean public HmilyTransactionBootstrap hmilyTransactionBootstrap(HmilyInitService hmilyInitService){ HmilyTransactionBootstrap hmilyTransactionBootstrap = new HmilyTransactionBootstrap(hmilyInitService); hmilyTransactionBootstrap.setSerializer(env.getProperty("org.dromara.hmily.serializer")); hmilyTransactionBootstrap.setRecoverDelayTime(Integer.parseInt(env.getProperty("org.dromara.hmily.recoverDelayTime"))); hmilyTransactionBootstrap.setRetryMax(Integer.parseInt(env.getProperty("org.dromara.hmily.retryMax"))); hmilyTransactionBootstrap.setScheduledDelay(Integer.parseInt(env.getProperty("org.dromara.hmily.scheduledDelay"))); hmilyTransactionBootstrap.setScheduledThreadMax(Integer.parseInt(env.getProperty("org.dromara.hmily.scheduledThreadMax"))); hmilyTransactionBootstrap.setRepositorySupport(env.getProperty("org.dromara.hmily.repositorySupport")); hmilyTransactionBootstrap.setStarted(Boolean.parseBoolean(env.getProperty("org.dromara.hmily.started"))); HmilyDbConfig hmilyDbConfig = new HmilyDbConfig(); hmilyDbConfig.setDriverClassName(env.getProperty("org.dromara.hmily.hmilyDbConfig.driverClassName")); hmilyDbConfig.setUrl(env.getProperty("org.dromara.hmily.hmilyDbConfig.url")); hmilyDbConfig.setUsername(env.getProperty("org.dromara.hmily.hmilyDbConfig.username")); hmilyDbConfig.setPassword(env.getProperty("org.dromara.hmily.hmilyDbConfig.password")); hmilyTransactionBootstrap.setHmilyDbConfig(hmilyDbConfig); return hmilyTransactionBootstrap; }
Add annotation on startup class
@ComponentScan({"org.dromara.hmily"})
c) Caller (bank1) implementation
Code implementation: AccountInfoServiceImpl
try: try Idempotent check try Suspension treatment Check that the balance is sufficient to deduct the amount Deduction amount confirm: empty cancel cancel Idempotent check cancel Null rollback processing Increase available balance
Note: when calling bank2 remotely, annotate @ Hmily on the interface called by feign
d) Participant (bank2) implementation
Code implementation: AccountInfoServiceImpl
try: empty confirm: confirm Idempotent check Formal increase amount cancel: empty
e) Summary
If the TCC transaction processing flow is compared with 2PC two-phase commit, 2PC is usually at the cross database DB level , and TCC is handled at the application level , It needs to be implemented through business logic. The implementation of this distributed transaction advantage The is that the application can define the granularity of data operation by itself, so that Reduce lock conflicts and improve throughput It's possible.
and deficiencies It lies in the application Very invasive , Each branch of business logic needs to implement three operations: try, confirm and cancel. In addition, its It is also difficult to realize , Different rollback strategies need to be implemented according to different failure causes such as network status and system failure.
5, Reliable message final consistency of solution
Project source code: cloud DTX txmsg
1. What is the final consistency of reliable messages
The reliable message final consistency scheme refers to when The transaction initiator executes the local transaction and issues a message , Transaction participants (message consumers) must be able to receive messages and process transactions successfully. This scheme emphasizes that as long as messages are sent to transaction participants, the final transaction must be consistent.
Problems to be solved for reliable messages:
- Atomicity of local transactions and message sending
- //First message if the database operation is wrong, the message has been sent begin transaction; // 1. Send MQ //2. Database operation commit transation; // If the database times out, the database rolls back, but the message may have been sent begin transaction; // 1. Database operation / / 2. Sending MQ commit transation;
- Reliability of message accepted by transaction participants
- Transaction participants must be able to receive messages from the message queue. If receiving messages fails, they can receive messages repeatedly.
- The problem of repeated consumption of messages
- Due to the existence of network 2, if a consumption node times out but the consumption is successful, the message middleware will deliver the message repeatedly, resulting in repeated consumption of the message. To solve the problem of repeated consumption of messages, it is necessary to realize the method idempotency of transaction participants.
2. RocketMQ transaction message scheme
- Producer Send transaction message : The producer (MQ sender) sends a transaction message to the MQ Server, which marks the message status as Prepared( Ready state ), Note that the message consumer (MQ subscriber) is Unable to consume Yes.
- MQ Server Response message Sending succeeded: when MQ server receives the message sent to by Producer, it responds that sending succeeded, indicating that MQ has received the message.
- Producer execution Local transaction : The producer side executes business code logic through Local database transaction control .
- Message delivery : If Producer local transaction Successful execution Automatically sent to MQServer commit Message. At this time, the MQ subscriber (point service) is the normal consumption message; If Producer local transaction Execution failed Automatically sent to MQServer rollback After receiving the rollback message, the MQ Server will delete the "add points message". The MQ subscriber (points service) consumes the message, If the consumption is successful, respond ack to MQ , Otherwise, the message will be received repeatedly. Here ack will respond automatically by default, that is, when the program is running normally, ACK will be responded automatically.
- Transaction backcheck : If the Producer side is executed During a local transaction, the execution end hangs or times out , MQ Server will keep asking Other producers in the same group to get the transaction execution status , This process is called transaction backcheck. MQ Server will decide whether to post messages according to the transaction backcheck results.
3. RocketMQ implements the final consistency transaction of reliable messages
a)SQL
bank1
CREATE DATABASE `bank1` CHARACTER SET 'utf8' COLLATE 'utf8_general_ci'; DROP TABLE IF EXISTS `account_info`; CREATE TABLE `account_info` ( `id` BIGINT (20) NOT NULL AUTO_INCREMENT, `account_name` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Name of head of household', `account_no` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Bank card No', `account_password` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Account password', `account_balance` DOUBLE NULL DEFAULT NULL COMMENT 'account balance', PRIMARY KEY (`id`) USING BTREE ) ENGINE = INNODB AUTO_INCREMENT = 5 CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic; INSERT INTO `account_info` VALUES ( 2, 'Zhang San's account', '1', '', 10000 ); DROP TABLE IF EXISTS `de_duplication`; CREATE TABLE `de_duplication` ( `tx_no` VARCHAR (64) COLLATE utf8_bin NOT NULL, `create_time` datetime (0) NULL DEFAULT NULL, PRIMARY KEY (`tx_no`) USING BTREE ) ENGINE = INNODB CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic; View Code
bank2
CREATE DATABASE `bank2` CHARACTER SET 'utf8' COLLATE 'utf8_general_ci'; DROP TABLE IF EXISTS `account_info`; CREATE TABLE `account_info` ( `id` BIGINT (20) NOT NULL AUTO_INCREMENT, `account_name` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Name of head of household', `account_no` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Bank card No', `account_password` VARCHAR (100) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL COMMENT 'Account password', `account_balance` DOUBLE NULL DEFAULT NULL COMMENT 'account balance', PRIMARY KEY (`id`) USING BTREE ) ENGINE = INNODB AUTO_INCREMENT = 5 CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic; INSERT INTO `account_info` VALUES ( 3, 'Li Si's account', '2', NULL, 0 ); CREATE TABLE `de_duplication` ( `tx_no` VARCHAR (64) COLLATE utf8_bin NOT NULL, `create_time` datetime (0) NULL DEFAULT NULL, PRIMARY KEY (`tx_no`) USING BTREE ) ENGINE = INNODB CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Dynamic; View Code
b) Installing RocketMQ
c) Engineering configuration
maven
<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-spring-boot-starter</artifactId> <version>2.0.2</version> </dependency>
properties configuration
rocketmq.producer.group = producer_bank2 rocketmq.name‐server = 127.0.0.1:9876
d)bank1
Service: AccountInfoServiceImpl
//Two methods //1. Send a transfer message to mq //2. Update the account and deduct the amount (ensure idempotency through transaction id)
Controller: AccountInfoController
//Generate the transaction id and call the messaging interface of the service
message: ProducerTxmsgListener
//There are two methods, executelocetransaction and checkLocalTransaction //The callback method after the transaction message is sent. At this time, ensure the local transaction, call the Service to deduct the amount, and change the message to COMMIT (consumable state). If an exception is caught, change the message to ROLLBACK //Transaction backcheck. Query whether the caller has processed it. If it has been processed, the message needs to be modified to COMMIT consumable. Otherwise, it is in the UNKOWN state.
e)bank2
Service: AccountInfoServiceImpl
//Update account bank2 and increase the amount. (ensure idempotency through transaction id)
message: TxmsgConsumer
//Listen to the message topic sent by bank1 and call Service to increase the amount
4. Summary
The ultimate consistency of reliable messages is Ensure that messages are delivered from the producer to the consumer through message middleware In this case, RocketMQ is used as the message middleware. RocketMQ mainly solves two functions:
- Atomicity of local transactions and message sending.
- Reliability of messages received by transaction participants.
Reliable message final consistency transaction fit Scenarios with long execution cycle and low real-time requirements . After the introduction of message mechanism, synchronous transaction operations become message based asynchronous The operation avoids the influence of synchronous blocking operation in distributed transactions, and realizes the synchronization of two services decoupling .
6, Best effort notification of Solutions
Source code: cloud DTX notify
1. What is best effort notification
The initiating party notifies through a certain mechanism Make every effort to inform the receiver of the business processing results .
- There is a certain Message repetition notification mechanism . Because the party receiving the notification may not receive the notification, there must be a certain mechanism to repeat the notification for the message.
- Message proofreading mechanism . If the receiver has not been notified as far as possible, or the receiver wants to consume the message again after consuming it, you can The receiving party shall actively query the notifying party for messages Information to meet needs.
2. Similarities and differences between best effort notification and reliable message consistency
- Different ideas: reliable message consistency, initiation The notifying party needs to ensure that the message is sent , And send the message to the receiving notifier. The reliability of the message is guaranteed by the initiating notifier. The initiating notifier tries its best to notify the receiving notifier of the business processing result, but the message may not be received. At this time, the receiving notifier needs to actively call the interface of the initiating notifier to query the business processing result, The key to the reliability of the notice lies in the party receiving the notice .
- Different business scenarios: reliable message consistency focuses on Transaction consistency in the transaction process , Complete the transaction asynchronously. Best effort notification focuses on Post transaction notification transaction , Inform the trading results reliably.
- Different technical solutions: reliable message consistency needs to be solved Message consistency from sending to receiving , That is, the message is sent and received; the best effort notification cannot guarantee the consistency of the message from sending to receiving, but only provides the reliability mechanism of message receiving. The reliability mechanism is, Try your best to notify the receiver of the message , When the message cannot be received by the receiver, the The receiver actively queries the message (business processing results).
3. Solution
a) Solution 1:
Specific process:
- The originating notifier sends the notification to MQ. The notification is sent to MQ using the normal message mechanism.
- The receiving party listens to MQ.
- The receiving notifier receives the message, and the service processing is completed to respond to the ack.
- If the party receiving the notice If there is no response to ack, MQ will repeat the notification . MQ will gradually increase the notification interval according to the interval of 1min, 5min, 10min, 30min, 1h, 2h, 5h and 10h (if MQ adopts rocketMq, it can be configured in the broker) until the upper limit of the time window required by the notification is reached.
- The receiving party can check the consistency of the message through the message checking interface.
b) Solution 2:
Unlike option 1 The application sends notifications to the recipient , As shown below:
Specific process:
- The initiating notifier sends the notification to MQ: the transaction message guarantee in the reliable message consistency scheme is used Local transactions and atomicity of messages , Finally, the notification will be sent to MQ first.
- The notifier listens to MQ and receives messages from MQ. If the notifier does not respond to ack, MQ will repeat the notification.
- Notification procedure Through Internet interface protocol (such as http, webservice) Call the receive notification scheme interface , The notification is completed. If the notifier calls the notification scheme interface successfully, the notification is successful, that is, the consumption of MQ messages is successful, and MQ will no longer deliver notification messages to the notifier.
- The receiving party can check the consistency of the message through the message checking interface.
c) Comparison of two schemes
- In scheme 1, the receiving notifier interfaces with MQ, that is, the receiving notification scheme listens to MQ. This scheme mainly Notification between application and internal application .
- In scheme 2, the notifier interfaces with MQ. The notifier listens to MQ. After receiving MQ messages, the notifier calls the receiving notifier through the Internet interface protocol. This scheme is mainly applied to Notification between external applications , For example, Alipay and WeChat pay the result notice.
4. Best effort notification
a)sql
bank1_pay.sql
CREATE DATABASE /*!32312 IF NOT EXISTS*/`bank1_pay` /*!40100 DEFAULT CHARACTER SET utf8 */; USE `bank1_pay`; /*Table structure for table `account_pay` */ DROP TABLE IF EXISTS `account_pay`; CREATE TABLE `account_pay` ( `id` varchar(64) COLLATE utf8_bin NOT NULL, `account_no` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'account number', `pay_amount` double DEFAULT NULL COMMENT 'Recharge balance', `result` varchar(20) COLLATE utf8_bin DEFAULT NULL COMMENT 'Recharge result:success,fail', PRIMARY KEY (`id`) USING BTREE ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC; /*Data for the table `account_pay` */ insert into `account_pay`(`id`,`account_no`,`pay_amount`,`result`) values ('5678ef0a-1ff0-4cfd-97ac-640d749d596f','1',2,'success'),('7d7d469c-f100-4066-b927-014c0c3aa010','1',2,'success'),('947fafad-c19c-46bc-b0f0-43703a124fd4','1',2,'success'); View Code
bank1.sql
CREATE DATABASE /*!32312 IF NOT EXISTS*/`bank1` /*!40100 DEFAULT CHARACTER SET utf8 */; USE `bank1`; /*Table structure for table `account_info` */ DROP TABLE IF EXISTS `account_info`; CREATE TABLE `account_info` ( `id` bigint(20) NOT NULL AUTO_INCREMENT, `account_name` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Name of head of household', `account_no` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Bank card No', `account_password` varchar(100) COLLATE utf8_bin DEFAULT NULL COMMENT 'Account password', `account_balance` double DEFAULT NULL COMMENT 'account balance', PRIMARY KEY (`id`) USING BTREE ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC; /*Data for the table `account_info` */ insert into `account_info`(`id`,`account_name`,`account_no`,`account_password`,`account_balance`) values (2,'Zhang San','1',NULL,1000); /*Table structure for table `de_duplication` */ DROP TABLE IF EXISTS `de_duplication`; CREATE TABLE `de_duplication` ( `tx_no` varchar(64) COLLATE utf8_bin NOT NULL, `create_time` datetime DEFAULT NULL, PRIMARY KEY (`tx_no`) USING BTREE ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin ROW_FORMAT=DYNAMIC; View Code
b) Engineering configuration
Consistency between basic configuration and reliable message
c) pay payer
Service: AccountPayServiceImpl
//Two methods //1. Insert the recharge record. Generate the transaction id and send the transaction id and recharge information to the MQ queue //2. Query recharge records. Provides a query to the caller.
Controller: AccountPayController
//Directly call the method in the Service to insert the recharge record
d)bank1
Service: AccountInfoServiceImpl
//Two methods //1. Update the account amount. The idempotency of the update is guaranteed according to the transaction id. //2. Remotely call pay to query the recharge result. If the status changes, update the current account at the same time.
message: NotifyMsgListener
//Listen for messages. Update account amount by calling Service, idempotent update.
Controller: AccountInfoController
//Query recharge result of calling Service
5. Summary
The best effort notification scheme is in distributed transactions The one with the lowest requirements for consistency , It is applicable to some businesses with low final consistency and time sensitivity; The best effort notification scheme needs to realize the following functions:
- Message repetition notification mechanism.
- Message proofreading mechanism. Actively call the interface to query and modify.
7, Comparison of four distributed transactions
2PC: The biggest criticism is one Blocking protocol . RM needs to wait for TM's decision after executing branch transactions. At this time, the service will block and lock resources. Due to its blocking mechanism and high worst-case time complexity, this design can not meet the needs of expansion with the increase of the number of services involved in transactions, and it is difficult to be used in distributed services with high concurrency and long-running transactions.
TCC : If the TCC transaction processing flow is compared with 2pc two-phase commit, 2PC Usually in Cross database DB level, and TCC is handled at the application level , It needs to be implemented through business logic. The advantage of this distributed transaction implementation is that it allows applications to define the granularity of data operations Reduce lock conflicts and improve throughput It's possible. The disadvantage is that it is very intrusive to applications. Each branch of business logic needs to implement three operations: try, confirm and cancel. In addition, its It is also difficult to realize , Different rollback strategies need to be implemented according to different failure causes such as network status and system failure. Typical usage scenarios: login, coupon, etc.
Reliable message final consistency transaction: fit Scenarios with long execution cycle and low real-time requirements . After the introduction of message mechanism, the synchronous transaction operation becomes asynchronous operation based on message execution, which avoids the influence of synchronous blocking operation in distributed transactions, and realizes the decoupling of the two services. Typical usage scenarios: Register to send points and login to send coupons Wait.
Best effort notification: It is one of the lowest requirements in distributed transactions, and is suitable for some services with low final consistency and time sensitivity; The initiating party is allowed to handle the business failure and actively handle the failure after the receiving party receives the notice. No matter how the initiating party handles the result, it will not affect the subsequent processing of the receiving party; The initiating notifier shall provide the query execution interface for Receiving party's proofreading results . Typical usage scenarios: Bank notice and payment result Notice, etc.
Original link:
http://www.cnblogs.com/bbgs-xc/p/14456917.html
If you think this article is helpful to you, pay attention to and support it!