Test RabbitMQ message response (manual gear, automatic gear)

concept

It may take some time for a consumer to complete a task. What happens if one of the consumers processes a long task and only completes part of it, and suddenly it hangs up. Once RabbitMQ delivers a message to the consumer, it immediately marks the message for deletion. In this case, a consumer suddenly hangs up, and we will lose the message being processed. And the subsequent message sent to the consumer because it cannot be received.

Why is there this concept?

In order to ensure that the message is not lost during sending, rabbitmq introduces a message response mechanism. The message response is: after receiving and processing the message, the consumer tells rabbitmq that it has been processed, and rabbitmq can delete the message.

Automatic response

The message is considered to have been successfully transmitted immediately after it is sent. This mode requires a trade-off between high throughput and data transmission security, because in this mode, if the connection or channel is closed on the consumer's side before the message is received, the message will be lost. On the other hand, of course, this mode can deliver overloaded messages on the consumer's side, There is no limit on the number of messages delivered. Of course, this may cause consumers to receive too many messages that are too late to process, resulting in the backlog of these messages, eventually running out of memory, and finally these consumer threads are killed by the operating system, Therefore, this model is only applicable when consumers can process these messages efficiently and at a certain rate.

Method of message response

Channel.basicAck //(for positive confirmation)
//RabbitMQ knows the message and processes it successfully. It can be discarded
Channel.basicNack  //(for negative confirmation)
Channel.basicReject //(for negative confirmation)
//Compared with Channel.basicNack, there is one less parameter (Multiple) that does not process the message. It is rejected directly and can be discarded

Multiple explanation
The advantage of manual response is that it can respond in batches and reduce network congestion

true and false of multiple mean different things

true means to batch respond to messages that are not answered on the channel
For example, if there are messages 5, 6, 7 and 8 on the channel that transmit the tag, and the current tag is 8, then these unacknowledged messages of 5-8 will be confirmed to receive the message response.

false compared with the above, only messages 5, 6 and 7 with tag=8 will be answered, and the three messages will not be confirmed to receive the message response.

Message auto rejoin

If the consumer loses the connection for some reason (its channel has been closed, the connection has been closed or the TCP connection has been lost), resulting in the message not sending ACK confirmation, RabbitMQ will understand that the message has not been fully processed and will queue it again. If other consumers can handle it at this time, it will soon redistribute it to another consumer. In this way, even if a consumer dies occasionally, it can be ensured that no message will be lost.

Implementation of message manual response

The default message adopts automatic response, so we need to change the automatic response to manual response in order to avoid loss in the process of message consumption.

Producer code

import com.rabbitmq.client.Channel;
import com.xiang.rabbitmq.util.RabbitMqUtils;

import java.util.Scanner;

/**
 * producer
 * Message manual answer
 */
public class Produce {

    public static final String QUEUE_NAME = "ack_queue";

    public static void main(String[] args) throws Exception {

        Channel channel = RabbitMqUtils.getChannel();
        // Declaration queue
        channel.queueDeclare(QUEUE_NAME,false,false,false,null);

        //Receive information from the console
        Scanner scanner = new Scanner(System.in);
        while (scanner.hasNext()){
            String message = scanner.next();
            channel.basicPublish("",QUEUE_NAME,null,message.getBytes("UTF-8"));
            System.out.println("PRODUCE Send message complete:"+message);
        }

    }
}

Consumer code

consumer

package com.xiang.rabbitmq.three;

import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;
import com.rabbitmq.client.Delivery;
import com.xiang.rabbitmq.util.RabbitMqUtils;

import java.util.concurrent.TimeUnit;

/**
 * consumer
 * The message is answered manually, the message is not lost, and the message is re queued
 */
public class Consumer {

    public static final String QUEUE_NAME = "ack_queue";

    public static void main(String[] args) throws Exception {

        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("[Q1 Waiting to receive message]");
        DeliverCallback deliverCallback = (String consumerTag, Delivery message)->{
            try {
                Thread.sleep(TimeUnit.SECONDS.toSeconds(2)); // Sleep for 2s
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("Received message:" + new String(message.getBody(),"UTF-8"));
            /**
             * Manual answer
             * 1.long deliveryTag, Message tag
             * 2 boolean multiple  Batch response
              */
            channel.basicAck(message.getEnvelope().getDeliveryTag(), false);
        };
        CancelCallback cancelCallback = (consumerTag)->{
            System.out.println("The consumer canceled the consumption message");
        };


        boolean autoAck = false;
        channel.basicConsume(QUEUE_NAME,autoAck,deliverCallback,cancelCallback);

    }
}

Consumer 01

package com.xiang.rabbitmq.three;

import com.rabbitmq.client.CancelCallback;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DeliverCallback;
import com.rabbitmq.client.Delivery;
import com.xiang.rabbitmq.util.RabbitMqUtils;

import java.util.concurrent.TimeUnit;

/**
 * consumer
 * The message is answered manually, the message is not lost, and the message is re queued
 */
public class Consumer01 {

    public static final String QUEUE_NAME = "ack_queue";

    public static void main(String[] args) throws Exception {

        Channel channel = RabbitMqUtils.getChannel();
        System.out.println("[Q2 Waiting to receive message]");
        DeliverCallback deliverCallback = (String consumerTag, Delivery message)->{
            try {
                Thread.sleep(TimeUnit.SECONDS.toSeconds(30)); // Sleep for 30s
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("Received message:" + new String(message.getBody(),"UTF-8"));
            /**
             * Manual answer
             * 1.long deliveryTag, Message tag
             * 2 boolean multiple  Batch response
             */
            channel.basicAck(message.getEnvelope().getDeliveryTag(), false);
        };
        CancelCallback cancelCallback = (consumerTag)->{
            System.out.println("The consumer canceled the consumption message");
        };


        boolean autoAck = false;
        channel.basicConsume(QUEUE_NAME,autoAck,deliverCallback,cancelCallback);

    }
}

Manual response experiment demonstration

Start program
In the experimental step Q2, the message processing time is 30s. After receiving the message in Q2, we simulate the Q2 consumer system fault in 30s to see whether the message is consumed? Whether to rejoin the team.

The producer sends two messages

Theoretically, Q1 and Q2 process a message respectively. After Q2 receives the message, there is a processing time of 30s. When processing, the system suddenly hangs up. Where is the message


It was found that the messages were consumed by Q1.

conclusion

After the sender sends the message test001, the Q2 consumer is stopped after sending the message, which is supposed to process the message. However, due to its long processing time, Q2 is stopped before it is processed, that is, Q2 has not executed the ack code. At this time, it will be seen that the message is received by Q1, indicating that the message test002 is re queued, It is then assigned to Q1, which can process messages

Tags: Java RabbitMQ message queue

Posted on Thu, 02 Dec 2021 15:51:23 -0500 by ikelove