Replacing scheduled tasks with delay queues

§ 1 RabbitMQ delay queue

RabbitMQ delay queue is mainly realized by TTL (Time to Live) and dead letter exchange (Dead Letter Exchanges) of messages.

It involves two queues, one for sending messages and one for forwarding destination queues after message expiration.

 

In this case, define two sets of exchange and queue.

agentpayquery1exchange - agentpayquery1queue (routingkey is delay)
agentpayquery2exchange - agentpayquery2queue (routingkey is delay)
agentpayquery1queue is a buffered queue, and messages are expired and routed to agentpayquery2queue

 

 

§ 2 producers

Producer configuration:

<!-- Connection service configuration -->
<rabbit:connection-factory
        id="connectionFactoryProducer"
        addresses="${mq.ip}"    //192.168.40.40:5672
        username="${username}"
        password="${password}"
        channel-cache-size="${cache.size}"
        publisher-confirms="${publisher.confirms}"
        publisher-returns="${publisher.returns}"
        virtual-host="/"
/>

<!--========================Outgoing query delay queue configuration begin =========================-->
<rabbit:queue id="agentpayquery2queue" durable="true" auto-delete="false" exclusive="false" name="agentpayquery2queue"/>
<rabbit:direct-exchange name="agentpayquery2exchange" durable="true" auto-delete="false" id="agentpayquery2exchange">
    <rabbit:bindings>
        <rabbit:binding queue="agentpayquery2queue" key="delay" />
    </rabbit:bindings>
</rabbit:direct-exchange>


<rabbit:queue id="agentpayquery1queue" durable="true" auto-delete="false" exclusive="false" name="agentpayquery1queue" >
    <rabbit:queue-arguments>
        <entry key="x-dead-letter-exchange" value="agentpayquery2exchange"/>
    </rabbit:queue-arguments>
</rabbit:queue>
<rabbit:direct-exchange name="agentpayquery1exchange" durable="true" auto-delete="false" id="agentpayquery1exchange">
    <rabbit:bindings>
        <rabbit:binding queue="agentpayquery1queue" key="delay" />
    </rabbit:bindings>
</rabbit:direct-exchange>

<!--Definition RabbitTemplate Example-->
<rabbit:template id="agentpayQueryMsgTemplate"
                 exchange="agentpayquery1exchange"  routing-key="delay"
                 queue="agentpayquery1queue"
                 connection-factory="connectionFactoryProducer" message-converter="mqMessageConverter"
                 mandatory="true"
                 confirm-callback="publisherConfirmsReturns" return-callback="publisherConfirmsReturns"/>
<!--========================Outgoing query delay queue configuration end =========================-->

 

 

The producer information enters the team:

import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageDeliveryMode;
import org.springframework.amqp.core.MessagePostProcessor;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class AgentpayQueryProducer {

    private static final Logger log = LogManager.getLogger(AgentpayQueryProducer.class.getSimpleName());

    @Autowired
    private RabbitTemplate agentpayQueryMsgTemplate;

    public void sendDelay(String message, int delaySeconds) {
        String expiration = String.valueOf(delaySeconds * 1000);
        agentpayQueryMsgTemplate.convertAndSend((Object) message, new MessagePostProcessor() {
            @Override
            public Message postProcessMessage(Message message)
                    throws AmqpException {
                message.getMessageProperties().setExpiration(expiration);
                message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
                log.info("Payment query data joining the team:{}", new String(message.getBody()));
                return message;
            }
        });
    }
}

 

 

3 consumers

The configuration of the consumer end is no other:

<!-- Connection service configuration  channel-cache-size="25" -->
<rabbit:connection-factory id="connectionFactory"
                           addresses="${mq.ip}"
                           username="${username}"
                           password="${password}" />

<bean id="agentpayQueryConsumer" class="com.emaxcard.rpc.payment.service.impl.batchpay.AgentpayQueryConsumer" />

<!-- queue litener  Observe the listening mode. When a message arrives, it will notify the listener on the corresponding queue-->
<rabbit:queue id="agentpayquery2queue" durable="true" auto-delete="false" exclusive="false" name="agentpayquery2queue" />

<rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto"

                           max-concurrency="20"
                           concurrency="10"
                           prefetch="10">
    <rabbit:listener ref="agentpayQueryConsumer" queues="agentpayquery2queue" />
</rabbit:listener-container>

 

Message consumption:

import com.alibaba.fastjson.JSON;
import com.emaxcard.enums.BatchPayStatus;
import com.emaxcard.exceptions.ResponseException;
import com.emaxcard.payment.vo.PaymentRecord;
import com.emaxcard.rpc.payment.model.PaymentRecordModel;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageListener;
import org.springframework.beans.factory.annotation.Autowired;

public class AgentpayQueryConsumer implements MessageListener {

    private static final Logger log = LogManager.getLogger();

    @Autowired
    QueryGatewayService queryGatewayService;
    @Autowired
    AgentpayQueryProducer agentpayQueryProducer;

    @Override
    public void onMessage(Message message) {
        String mqMsg = new String(message.getBody());
        log.info("Outgoing query data outgoing:{}", mqMsg);
        PaymentRecord paymentRecordModel;
        try {
            paymentRecordModel = JSON.parseObject(mqMsg, PaymentRecord.class);
        } catch (Exception ex) {
            log.info("Message format is not PaymentRecordModel,End.");
            return;
        }

        try {
            BatchPayStatus payStatus = queryGatewayService.queryGateway(paymentRecordModel);

            // Non final status, continue to put into delay queue
            if (BatchPayStatus.SUCCESS != payStatus && BatchPayStatus.FAILED != payStatus) {
                if (BatchPayStatus.NOTEXIST == payStatus) {
                    log.info("The query result is{},No longer deal with", payStatus);
                } else {
                    agentpayQueryProducer.sendDelay(mqMsg, 10);
                }
            }
        } catch (Exception ex) {
            if (ex instanceof ResponseException) {
                log.info("Transfer inquiries{},paymentId{},Processing error:{}",
                        paymentRecordModel.getTransNo(), paymentRecordModel.getPaymentId(), ex.getMessage());
            } else {
                log.error("Handling message exception:", ex);
            }
        }

    }
}

 

 

§ 4 pay attention to using delay queue

1. Because it is a queue, even if a message expires earlier than other messages in the same queue, the messages that expire earlier will not enter the dead letter queue first. They still let consumers consume in the order of warehousing. If the expiration time of the first incoming message is 1 hour, the consumers in the dead letter queue may wait 1 hour to receive the first message.

2. Once the message with no expiration time is set in the buffer queue, the whole queue will be blocked. Consumers can't consume messages. As you can see from the log, all the printouts are blockingqueueconsumers.

 

 

 

Get messages Ack Mode select "ACK message request false" to consume messages

Tags: Java RabbitMQ JSON

Posted on Tue, 03 Dec 2019 19:32:12 -0500 by teejayuu