RocketMQ custom message queue selector to ensure message order

How to ensure global order?

The simplest way to ensure global order is that there is only one queue in topic, which can ensure global order, but will anyone use it like this? Certainly not, because the performance, throughput and security will be very poor

Therefore, it is to ensure local order, not global order

Ensure orderly usage scenarios

For order payment, an order must come down in order. For example, it must be paid first, then marketed, and then sent to logistics. In this way, the order cannot be disordered

Or chat function: we all need to ensure the order of sending. It can't be said that you sent it first and then came after others

So this is to ensure local order

case

The producer sends 10 orders. There are six steps in each order. Each step will send a message. In the past, these six steps are required to be sequential

Consumers should ensure that the order of consumption is the same

All MQ S can only ensure that messages are ordered in one queue. If Kafka is used, it is Partition

RocketMQ guarantees local order, not global order

What is sequential local order? Partial order requires the cooperation of your producers and consumers

consumer

Note: when registering a Listener, registerMessageListener needs to use MessageListenerOrderly, and the extracted messages are orderly. If other messagelisteners, such as messagelistenercurrently, are used, the order cannot be guaranteed

package org.apache.rocketmq.example.ordermessage;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

public class Consumer {

    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name_3");
        consumer.setNamesrvAddr("zjj101:9876;zjj102:9876;zjj103:9876");
        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);

        consumer.subscribe("OrderTopicTest", "*");
        /**
         *The consumer registers a listener, MessageListenerOrderly,
         * Get it from the queue
         */
        consumer.registerMessageListener(new MessageListenerOrderly() {
            /***
             *
             * @param msgs
             * @param context
             * @return
             */
            @Override
            public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) {
                context.setAutoCommit(true);
                for (MessageExt msg : msgs) {
                    System.out.println("Received message content " + new String(msg.getBody()));
                }
                return ConsumeOrderlyStatus.SUCCESS; // Return successful consumption ID
            }
        });

        // Messagelistenercurrently is taken indiscriminately, and the consumption order cannot be guaranteed

//        This will not guarantee the order of final consumption.
//        consumer.registerMessageListener(new MessageListenerConcurrently() {
//            @Override
//            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
//                for(MessageExt msg:msgs){
//                    System.out.println("received message content" + new String(msg.getBody()));
//                }
//                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
//            }
//        });

        consumer.start();
        System.out.printf("Consumer Started.%n");
    }

}

Custom message queue selector

I wrote the notes in detail, so I won't explain more,

In addition, you should pay attention to the fact that the String may have negative numbers when hashCode

Why is there a negative number after the string hashCode is completed? Read this blog: https://zjj1994.blog.csdn.net/article/details/120875055

package org.apache.rocketmq.example.ordermessage;

import org.apache.rocketmq.client.producer.MessageQueueSelector;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageQueue;

import java.util.List;

/**
 * Custom message queue selector
 */
public class MyMessageQueueSelector implements MessageQueueSelector {

    /**
     * Select queue
     *
     * @param mqs All messagequeues under this topic
     * @param msg Messages sent
     * @param arg This parameter is passed from the third parameter when the producer calls the send method
     * @return
     */
    @Override
    public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {


        // orderId takes the Size of MessageQueue and obtains an index, which ensures that messages in the same Order are stored in the same queue

        String orderId = (String) arg; // Get the passed orderId

        // Perform hash operation
        //Below & integer.max_ The purpose of value is to copy the hashCode, and the negative number comes out
        int orderIdHashCode = orderId.hashCode() & Integer.MAX_VALUE;

        // The hashCode value and the length of the queue are taken as the remainder to get an integer
        int index = orderIdHashCode % mqs.size();

        // Get a queue through the remainder above and deliver messages to the queue, so as to ensure
        MessageQueue messageQueue = mqs.get(index);
        return messageQueue;
    }
}

producer

500 orders are generated. Each order has four steps, such as placing an order, paying, confirming receipt and evaluating. A message will be sent in each step, and the message cannot be out of order, that is, the messages that cannot be paid will come before the order message, so there is a business bug

package org.apache.rocketmq.example.ordermessage;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.common.RemotingHelper;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.io.UnsupportedEncodingException;
import java.util.UUID;

public class Producer {
    public static void main(String[] args) throws UnsupportedEncodingException {
        try {
            DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
            producer.setNamesrvAddr("zjj101:9876;zjj102:9876;zjj103:9876");
            producer.start();

            for (int i = 0; i < 500; i++) { // Generate 500 orders

                //Generate 50 orders, where the order number is replaced by uuid. In fact, the order id generation scheme of each company is different
                String orderId = UUID.randomUUID().toString();
                // Each order has four steps, such as placing an order, paying, confirming receipt and evaluating. Each step will send a message in the past, and the message cannot be out of order, that is, payment cannot come before placing an order
                for (int j = 1; j <= 4; j++) {


                    // Instantiate a message
                    Message msg =
                            new Message("OrderTopicTest", "orderTag", "KEY" + orderId,
                                    ("order Id:" + orderId + " step:" + j).getBytes(RemotingHelper.DEFAULT_CHARSET));

                    //Instantiate the message queue selector written by yourself
                    MyMessageQueueSelector myMessageQueueSelector = new MyMessageQueueSelector();

                    /*
                     * MessageQueueSelector Message queue selector is used to select the queue to which messages are sent
                     * @param Parameter 1: MSG message you sent
                     * @param Parameter 2: selector, message Queue selector,
                     * @param Parameter 3: args is passed to the parameter used by the message queue selector. This is the third parameter of the select method in myMessageQueueSelector
                     */
                    SendResult sendResult = producer.send(msg, myMessageQueueSelector, orderId);

                    System.out.printf("%s%n", sendResult);
                }
            }

            producer.shutdown();
        } catch (MQClientException | RemotingException | MQBrokerException | InterruptedException e) {
            e.printStackTrace();
        }
    }
}