In the actual project, the message consumption often fails, so it is necessary to resend the message. For example, after the failure of payment message consumption, N times of message retransmission reminder should be performed in different time periods.
This paper simulates the scene
- Message consumption succeeds when the amount is less than 100
- When the amount is greater than 100 and less than 200, three retransmissions will be performed, one second for the first time, two seconds for the second time and three seconds for the third time.
- When the amount is greater than 200, message consumption fails, and the message will be resend for 5 times, the first time for 1 second, the second time for 2 seconds, the third time for 3 seconds, the fourth time for 4 seconds, and the fifth time for 5 seconds. After five retries, the message automatically enters the dead letter queue and disappears after 60 seconds of the dead letter queue.
Code instance
Pay special attention to the comments in the code and configuration file. The instructions have been written in detail in the configuration file
pom package introduction
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.12.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.cloudstream</groupId> <artifactId>demo</artifactId> <version>0.0.1-SNAPSHOT</version> <name>demo</name> <description>Demo project for Spring Boot</description> <properties> <java.version>1.8</java.version> <spring-cloud.version>Greenwich.SR5</spring-cloud.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!-- ①Key configuration: Introduction stream-rabbit rely on--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-rabbit</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency> </dependencies> <dependencyManagement> <dependencies> <!-- ②Critical configuration: due to stream Is based on spring-cloud Yes, so we need to introduce --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> <repositories> <repository> <id>spring-snapshots</id> <name>Spring Snapshots</name> <url>https://repo.spring.io/snapshot</url> <snapshots> <enabled>true</enabled> </snapshots> </repository> <repository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> </repository> </repositories> </project>
to configure application.yml file
Pay attention to the indentation format of each configuration. Don't get it wrong
server: port: 8081 spring: application: name: stream-demo #rabbitmq connection configuration rabbitmq: host: 127.0.0.1 port: 5672 username: admin password: 123456 cloud: stream: bindings: #Message producer, delay in interface with DelayDemoTopic_ DEMO_ The value of the product variable is the same delay-demo-producer: #① Define switch name destination: demo-delay-queue #Message consumer, delay in interface with DelayDemoTopic_ DEMO_ Consumer variable has the same value delay-demo-consumer: #Define the switch name, which is consistent with ①, so that both sending and consumption point to a queue destination: demo-delay-queue #Grouping. This configuration can enable message persistence and solve the problem of repeated consumption in the cluster environment. #For example, if there is no such configuration, both A and B will receive the same message. If there is such A configuration, only one of them will receive the message group: delay-consumer-group consumer: #Maximum number of retries, 3 by default. Do not use the default. Here it is defined as 1. Our program controls the sending time and times maxAttempts: 1 rabbit: bindings: #Message producer, delay in interface with DelayDemoTopic_ DEMO_ The value of the product variable is the same delay-demo-producer: producer: #② Declare as delay queue delayedExchange: true #Message consumer, delay in interface with DelayDemoTopic_ DEMO_ Consumer variable has the same value delay-demo-consumer: consumer: #Declared as a delay queue, paired with the configuration of ② delayedExchange: true #Open dead letter queue autoBindDlq: true #Lifetime of messages in dead letter queue dlqTtl: 60000
Define queue channel
- Defining channels
/** * Define delay message channel */ public interface DelayDemoTopic { /** * Producer, corresponding to yml file configuration */ String DELAY_DEMO_PRODUCER = "delay-demo-producer"; /** * Consumer, corresponding to yml file configuration */ String DELAY_DEMO_CONSUMER = "delay-demo-consumer"; /** * Define message consumers, which are used when @ StreamListener listens for messages * @return */ @Input(DELAY_DEMO_CONSUMER) SubscribableChannel delayDemoConsumer(); /** * Define the message sender, which is used when sending messages * @return */ @Output(DELAY_DEMO_PRODUCER) MessageChannel delayDemoProducer(); }
- Bind channel
/** * Configuring the binding of messages * */ @EnableBinding(value = {DelayDemoTopic.class}) @Component public class MessageConfig { }
Message sending simulation
/** * send message */ @RestController public class SendMessageController { @Autowired DelayDemoTopic delayDemoTopic; @GetMapping("send") public Boolean sendMessage(BigDecimal money) throws JsonProcessingException { Message<BigDecimal> message = MessageBuilder.withPayload(money) //Set the delay time of the message, send for the first time, do not set the delay time, send directly .setHeader(DelayConstant.X_DELAY_HEADER,0) //Set the number of times the message has been retried, sent for the first time, set to 0 .setHeader(DelayConstant.X_RETRIES_HEADER,0) .build(); return delayDemoTopic.delayDemoProducer().send(message); } }
Message listening and processing
@Component @Slf4j public class DelayDemoTopicListener { @Autowired DelayDemoTopic delayDemoTopic; /** * Listen for messages in the delay message channel * @param message */ @StreamListener(value = DelayDemoTopic.DELAY_DEMO_CONSUMER) public void listener(Message<BigDecimal> message) { //Get retries int retries = (int)message.getHeaders().get(DelayConstant.X_RETRIES_HEADER); //Get message content BigDecimal money = message.getPayload(); try { String now = DateUtils.formatDate(new Date(),"yyyy-MM-dd HH:mm:ss"); //Simulation: if the amount is greater than 200, the message cannot be consumed successfully; if the amount is greater than 100, retry 3 times; if the amount is less than 100, direct consumption succeeds if (money.compareTo(new BigDecimal(200)) == 1){ throw new RuntimeException(now+":The amount exceeds 200 and cannot be traded."); }else if (money.compareTo(new BigDecimal(100)) == 1 && retries <= 3) { if (retries == 0) { throw new RuntimeException(now+":If the amount exceeds 100, the consumption fails and will enter retry."); }else { throw new RuntimeException(now+":If the amount exceeds 100, the current" + retries + "Retries."); } }else { log.info("Message consumption succeeded!"); } }catch (Exception e) { log.error(e.getMessage()); if (retries < DelayConstant.X_RETRIES_TOTAL){ //Re queue messages MessageBuilder<BigDecimal> messageBuilder = MessageBuilder.fromMessage(message) //Set delay time for messages .setHeader(DelayConstant.X_DELAY_HEADER,DelayConstant.ruleMap.get(retries + 1)) //Set the number of times the message has been retried .setHeader(DelayConstant.X_RETRIES_HEADER,retries + 1); Message<BigDecimal> reMessage = messageBuilder.build(); //Resend message to delay queue delayDemoTopic.delayDemoProducer().send(reMessage); }else { //If the number of retries is exceeded, relevant processing (such as saving the database) will be done. If an exception is thrown, it will automatically enter the dead letter queue throw new RuntimeException("Exceeded maximum retries:" + DelayConstant.X_RETRIES_TOTAL); } } } }
Rule definition
Currently, it is written in a constant class. In the actual project, it is usually configured in the configuration file
public class DelayConstant { /** * Define the current number of retries */ public static final String X_RETRIES_HEADER = "x-retries"; /** * Define delay message, fixed value. Put this configuration in the header of the message, delay queue will be opened */ public static final String X_DELAY_HEADER = "x-delay"; /** * Define maximum retries */ public static final Integer X_RETRIES_TOTAL = 5; /** * Define retry rules in milliseconds */ public static final Map<Integer,Integer> ruleMap = new HashMap(){{ put(1,1000); put(2,2000); put(3,3000); put(4,4000); put(5,5000); }}; }
test
After the above configuration and implementation, the simulated retransmission scenario can be completed.
- Browser input http://127.0.0.1:8081/send?money=10, you can see the output in the console:
Message consumption succeeded!
- Browser input http://127.0.0.1:8081/send?money=110, you can see the output in the console:
2020-06-20 10:59:42: if the amount exceeds 100 and the consumption fails, it will enter retry. 2020-06-20 10:59:43: the amount exceeds 100, the first retry at present. 2020-06-20 10:59:45: if the amount exceeds 100, try again for the second time. 2020-06-20 10:59:48: the amount exceeds 100, which is the third retry at present. Message consumption succeeded!
- Browser input http://127.0.0.1:8081/send?money=110, you can see the output in the console:
matters needing attention
Because the delay queue is used in this article, you need to install the delay plug-in in rabbitMQ. For the specific installation method, you can see: Delay queue installation reference
Source code acquisition
All of the above examples are available through My GitHub Get the complete code