RabbitMQ learning (MQ learning length 2)

Learning of middleware MQ

Preface: the previous article has introduced the basic knowledge and simple application of RabbitMQ. This article introduces several advanced features of RabbitMQ
1, Reliable delivery of messages
1. It can be guessed from the name that this feature is mainly related to the producer, because delivery is easily associated with the process of sending messages to the switch after production and then pushing them to the queue. You can see the message delivery path of RabbitMQ as follows
Producer----->Brocker----->Exchange---->Queue---->Consumer
RabbitMQ provides us with two ways to control the reliable delivery of messages: confirm mode and return mode
If the message is from producer to exchange, a confirmCallback will be returned.
If the message fails to be delivered from exchange – > queue, a returnCallback will be returned. We will use these two callbacks to control the reliable delivery of messages
2. Next, we will explain how it makes use of these two features through code and text description
(1) Enable confirm mode:
1. Set publisher confirms = "true" of ConnectionFactory to enable confirmation mode.
2. Use rabbitTemplate.setConfirmCallback to set the callback function. When the message is sent to exchange, the confirm method is called back. Judge the ack in the method. If it is true, the sending succeeds. If it is false, the sending fails and needs to be processed.
3. Code: it mainly involves the test code and configuration file of the producer

//Test Confirm mode
@Test
public void testConfirm() {
    //Define callback
    rabbitTemplate.setConfirmCallback(new RabbitTemplate.ConfirmCallback() {
        /**
         *
         * @param correlationData Related configuration information
         * @param ack   exchange Whether the switch successfully received the message. true means success and false means failure
         * @param cause Failure reason
         */
        @Override
        public void confirm(CorrelationData correlationData, boolean ack, String cause) {
            System.out.println("confirm Method was executed....");

            //ack is true, indicating that the message has reached the switch
            if (ack) {
                //Received successfully
                System.out.println("Receive success message" + cause);
            } else {
                //Receive failed
                System.out.println("Receive failure message" + cause);
                //Do some processing to send the message again.
            }
        }
    });
    //Send message
    rabbitTemplate.convertAndSend("test_exchange_confirm", "confirm", "message Confirm...");
    //Perform sleep operation
    try {
        Thread.sleep(5000);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

<!-- definition rabbitmq connectionFactory -->
    <rabbit:connection-factory id="connectionFactory" host="${rabbitmq.host}"
                               port="${rabbitmq.port}"
                               username="${rabbitmq.username}"
                               password="${rabbitmq.password}"
                               virtual-host="${rabbitmq.virtual-host}"
                               publisher-confirms="true"
                               publisher-returns="true"
    />
    <!--Define management switches, queues-->
    <rabbit:admin connection-factory="connectionFactory"/>

    <!--definition rabbitTemplate Object operation can easily send messages in code-->
    <rabbit:template id="rabbitTemplate" connection-factory="connectionFactory"/>

    <!--Message reliability delivery (production side)-->
    <rabbit:queue id="test_queue_confirm" name="test_queue_confirm" auto-declare="true"></rabbit:queue>
    <!--Define switches and their bindings to queues-->
    <rabbit:direct-exchange name="test_exchange_confirm" auto-declare="true">
        <rabbit:bindings>
            <rabbit:binding queue="test_queue_confirm" key="confirm"></rabbit:binding>
        </rabbit:bindings>
    </rabbit:direct-exchange>

(2) Turn on returnCallback
1. Set publisher returns = "true" of ConnectionFactory to enable the return mode.
2. Use rabbitTemplate.setReturnCallback to set the return function. When the message fails to be routed from exchange to queue, if the rabbitTemplate.setMandatory(true) parameter is set, the message will be returned to the producer. And execute the callback function returnedMessage
3. Code: it mainly involves the test code and configuration file of the producer

	//Test return mode
    @Test
    public void testReturn() {
        //When the switch processing failure message mode is set to true, the message will be returned to the producer again when the message cannot reach the queue
        rabbitTemplate.setMandatory(true);
        //Define callback
        rabbitTemplate.setReturnCallback(new RabbitTemplate.ReturnCallback() {
            /**
             *
             * @param message   Message object
             * @param replyCode Error code
             * @param replyText error message
             * @param exchange  Switch
             * @param routingKey Routing key
             */
            @Override
            public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
                System.out.println("return Yes....");

                System.out.println("message:" + message);
                System.out.println("replyCode:" + replyCode);
                System.out.println("replyText:" + replyText);
                System.out.println("exchange:" + exchange);
                System.out.println("routingKey:" + routingKey);

                //handle
            }
        });
        //Send message
        rabbitTemplate.convertAndSend("test_exchange_confirm", "confirm", "message return...");
        //Perform sleep operation
        try {
            Thread.sleep(5000);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
xml The configuration is the same as the above configuration, mainly in configuration connectionFactory Properties set when

2, Consumer Ack
1. It is not difficult to guess from the name. This mainly involves consumers. It is used to ensure that messages in the queue are successfully consumed after being accepted by consumers. ack indicates that there are three confirmation methods after receiving a message:
Automatic confirmation: acknowledge = "none"
Manual confirmation: acknowledge = "manual"
Confirm according to abnormal conditions: acknowledge = "auto"
2. Add it and explain its function through code and text

	<!--Define listener container
      acknowledge="manual":Manual sign in
      prefetch="1":How many messages are captured each time
    -->
    <rabbit:listener-container connection-factory="connectionFactory" acknowledge="manual" prefetch="2"/>
@Component
public class AckListener implements ChannelAwareMessageListener {
    @Override
    public void onMessage(Message message, Channel channel) throws Exception {
        //1. Gets the id of the message
        long deliveryTag = message.getMessageProperties().getDeliveryTag();
        try {
            //2. Get message
            System.out.println("message:" + new String(message.getBody()));
            //3. Conduct business processing
            System.out.println("=====Conduct business processing====");
            //The simulation is abnormal
            int  i = 5/0;
            //4. Sign in messages
            channel.basicAck(deliveryTag, true);
        } catch (Exception e) {
            //Refuse to sign
             /*
            The third parameter: request: return to the queue. If it is set to true, the message will return to the queue, and the broker will resend the message to the consumer
             */
            channel.basicNack(deliveryTag, true, true);

        }
    }
}

The above code: when ack is configured as manual in xml, the consumption needs to be confirmed manually after the message consumption is completed. If an exception occurs during the process, you can use channel.basicNack(deliveryTag, true, true) to reject the signed message and return to the queue, so that the listener can listen to the message again for consumption. If the ACK configured in the xml is null, the message will be confirmed by default after being received by the consumer. At this time, if an exception occurs and the message has not been processed, the message will be lost.
3. Summary: in fact, in order to achieve the reliability of messages, both producers and consumers need to be configured to ensure that producer messages enter the queue and that consumer messages are successfully consumed. At the same time, it ensures persistence and high availability of broker s, and will not cause message loss due to hanging up an MQ.
3, Consumer end current limiting
This name cannot be understood. Generally speaking, it is worried about the insufficient consumption capacity of consumers, pulling too many messages at a time and exploding the consumer end. The prefetch attribute in the xml configuration of the Consumer Ack above is used to control the number of messages pulled each time.
4, TTL
The full name of TTL is Time To Live, that is, the expiration time can be set. This expiration can be set for queue expiration or message expiration. That is, after the expiration time is reached, the message will expire. It should be noted that there is a difference between queue expiration and setting message expiration. If the queue expires, all messages in the queue will expire. If the message expires, it will be recognized only when the message is at the head of the queue. If the expiration time is set for both, whichever is shorter.
5, Dead letter queue
1. At first, I heard that the dead letter queue seemed to be very tall, and I didn't know what to do. Later, I was asked in the interview how to prevent the loss of messages when there were too many production messages. Only then did I know that the dead letter queue can be used to solve the problem. What is the dead letter queue? In fact, it is also a queue, but the messages stored in it are different from those in the normal queue, which can be understood as abandoned messages. See the following explanation for specific conditions.
2. There are three situations when a message becomes a dead letter

  1. The queue message length reaches the limit;
  2. The consumer rejects the consumption message, basicNack/basicReject, and does not put the message back into the original target queue, request = false;
  3. The message expiration setting exists in the original queue, and the message arrival timeout is not consumed;

3. Next, the working mode of dead letter queue will be explained through pictures, codes and text

1. Normal switch and normal column binding (this is the normal process. The producer sends messages to the switch and then to the queue)
2. The normal queue is bound to the dead letter switch (if the message meets one of the three conditions of dead letter, it will enter the dead letter switch and then enter the dead letter queue)
3. The dead letter switch is bound to the dead letter queue (you can consume the messages of the dead letter queue by listening to the dead letter queue with a listener)

	<!--
        Dead letter queue:
            1. Declare a normal queue(test_queue_dlx)And switches(test_exchange_dlx)
            2. Declare dead letter queue(queue_dlx)Dead letter switch(exchange_dlx)
            3. Normal queue binding dead letter switch
                Set two parameters:
                    * x-dead-letter-exchange: Dead letter switch name
                    * x-dead-letter-routing-key: Sent to dead letter exchange routingkey
    -->
    <!--
        1. Declare a normal queue(test_queue_dlx)And switches(test_exchange_dlx)
    -->
    <rabbit:queue name="test_queue_dlx" id="test_queue_dlx">
        <!--3. Normal queue binding dead letter switch-->
        <rabbit:queue-arguments>
            <!--3.1 x-dead-letter-exchange: Dead letter switch name-->
            <entry key="x-dead-letter-exchange" value="exchange_dlx" />
            <!--3.2 x-dead-letter-routing-key: Sent to dead letter exchange routingkey-->
            <entry key="x-dead-letter-routing-key" value="dlx.hehe" />  <!--there value Why does it have to be this?-->
            <!--4.1 Set the expiration time of the queue ttl-->
            <entry key="x-message-ttl" value="10000" value-type="java.lang.Integer" />
            <!--4.2 Set the length limit of the queue max-length -->
            <entry key="x-max-length" value="10" value-type="java.lang.Integer" />
        </rabbit:queue-arguments>
    </rabbit:queue>
    <rabbit:topic-exchange name="test_exchange_dlx">
        <rabbit:bindings>
            <rabbit:binding pattern="test.dlx.#" queue="test_queue_dlx"></rabbit:binding>
        </rabbit:bindings>
    </rabbit:topic-exchange>
    
    <!--
       2. Declare dead letter queue(queue_dlx)Dead letter switch(exchange_dlx)
   -->
    <rabbit:queue name="queue_dlx" id="queue_dlx"></rabbit:queue>
    <rabbit:topic-exchange name="exchange_dlx">
        <rabbit:bindings>
            <rabbit:binding pattern="dlx.#" queue="queue_dlx"></rabbit:binding>
        </rabbit:bindings>
    </rabbit:topic-exchange>
// Consumer end
 <!--Define listener, dead letter queue-->
        <rabbit:listener ref="dlxListener" queue-names="queue_dlx"></rabbit:listener>

6, Delay queue
Delay queue means that the messages entering the queue will not be sent immediately, and there will be a delay. However, RabbitMQ does not have a delay queue, but it can be implemented through TTL + dead letter queue.
As shown in the figure
After the message enters the TTL (expired) queue, we actually do not listen to the messages in this queue. After the message expires, it enters the dead letter queue, and we listen to the dead letter queue. At this time, the received message has passed an expiration time extension.
7, Message idempotency
Message idempotency means that no matter how many times the message is consumed, the result is always the same. For example, each message carries a globally unique id. after receiving it, the message is stored in the local database. After each acceptance, the unique id is used to query the database to see whether it exists. If it exists, the previous result is returned or not processed. If it does not exist, a new message is being processed.

Here we are finished with the advanced features of RabbitMQ. The things sorted out are convenient for you to view in the future, so some words are too oral, and some points are not particularly detailed. If any watcher finds omissions or errors, you are welcome to correct them.

Tags: Java RabbitMQ

Posted on Wed, 01 Sep 2021 17:46:33 -0400 by ksas025