rabbitmq implementation of spring boot for message reliability

1. The producer module realizes message reliability through publisher confirm mechanism

1.1 producer module import rabbitmq related dependencies

<!--AMQP Dependency, including RabbitMQ-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<!--be used for mq Serialization and deserialization of messages-->
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
</dependency>

1.2 configure mq in the configuration file

spring.rabbitmq.host=10.128.240.183
spring.rabbitmq.port=5672
spring.rabbitmq.virtual-host=/
        
spring.rabbitmq.publisher-confirm-type=correlated
        
spring.rabbitmq.publisher-returns=true
        
spring.rabbitmq.template.mandatory=true
  • Publish confirm type: enable publisher confirm, with the following optional values
    • simple: the synchronization waits for the confirm result until it times out
    • correlated: asynchronous callback. ConfirmCallback is defined. The ConfirmCallback will be called back when mq returns the result
  • Publish returns: enable the publish return function. ReturnCallback can be defined
  • template.mandatory: defines the policy for message routing failure
    • true: call ReturnCallback
    • false: discard the message directly

1.3 define ReturnCallback (this callback is triggered when message delivery to queue fails)

  • Only one ReturnCallback can be configured for each RabbitTemplate.
  • When the message delivery fails, the processing logic defined in the producer's returnCallback will be called
  • You can configure this callback when the container starts
@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        // Gets the RabbitTemplate object
        RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
        // Configure ReturnCallback
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
            // Determine whether it is a delayed message
            Integer receivedDelay = message.getMessageProperties().getReceivedDelay();
            if (receivedDelay != null && receivedDelay > 0) {
                // Is a delayed message. Ignore this error message
                return;
            }
            // Log
            log.error("Failed to send message to queue, response code:{}, Failure reason:{}, Switch: {}, route key: {}, news: {}",
                     replyCode, replyText, exchange, routingKey, message.toString());
            // Resend the message if necessary
        });
    }
}

1.4 define ConfirmCallback (this callback is triggered when the message arrives at the switch)

  • You can specify a unified confirmation callback for redisTemplate
@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        // Gets the RabbitTemplate object
        RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
        // Configure ReturnCallback
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
            // Determine whether it is a delayed message
            Integer receivedDelay = message.getMessageProperties().getReceivedDelay();
            if (receivedDelay != null && receivedDelay > 0) {
                // Is a delayed message. Ignore this error message
                return;
            }
            // Log
            log.error("Failed to send message to queue, response code:{}, Failure reason:{}, Switch: {}, route key: {}, news: {}",
                     replyCode, replyText, exchange, routingKey, message.toString());
            // Resend the message if necessary
        });

        
        // Set up a unified confirm callback. ack=true as long as the message reaches the broker
        rabbitTemplate.setConfirmCallback(new RabbitTemplate.ConfirmCallback() {
            @Override
            public void confirm(CorrelationData correlationData, boolean b, String s) {
                System.out.println("This is a unified callback");
                System.out.println("correlationData:" + correlationData);
                System.out.println("ack:" + b);
                System.out.println("cause:" + s);
            }
        });
    }
}
  • Callbacks can also be customized for specific messages
    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Test
    public void testmq() throws InterruptedException {


        CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());

        correlationData.getFuture().addCallback(result->{
            if (result.isAck()) {
                // ACK
                log.debug("The message was successfully delivered to the switch! news ID: {}", correlationData.getId());
            } else {
                // NACK
                log.error("Message delivery to switch failed! news ID: {}", correlationData.getId());
                // Resend message
            }
        },ex->{
            // Log
            log.error("Message sending failed!", ex);
            // Resend message
        });
        rabbitTemplate.convertAndSend("example.direct","blue","hello,world",correlationData);
    }

2. Consumer module startup message confirmation

2.1 add configuration

# Manual ack messages do not use the default consumer acknowledgement
spring.rabbitmq.listener.simple.acknowledge-mode=manual
  • none: when ack is turned off, the message delivery is unreliable and may be lost
  • auto: similar to the transaction mechanism, nack is returned when an exception occurs, and the message is rolled back to mq. If there is no exception, ack is returned
  • manual: we specify when to return ack

2.2 in manual mode, the ack returned by the listener can be customized

@RabbitListener(queues = "order.release.order.queue")
@Service
public class OrderCloseListener {
 
    @Autowired
    private OrderService orderService;
 
 
    @RabbitHandler
    private void listener(OrderEntity orderEntity, Channel channel, Message message) throws IOException {
        System.out.println("Receive overdue order information and prepare to close the order" + orderEntity.getOrderSn());
 
        try {
            orderService.closeOrder(orderEntity);
            // If the second parameter is false, it means that only this message is confirmed. If true, it means to confirm multiple messages received at the same time
            channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
        } catch (Exception e) {
            // The second parameter, ture, means to rejoin the message to the queue
            channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
        }
    }
}

3. The consumer module failed to open the message retry mechanism

3.1 add configuration to configuration file, enable local retry

spring:
  rabbitmq:
    listener:
      simple:
        retry:
          enabled: true # Failed to open consumer. Try again
          initial-interval: 1000 # The waiting time for the first failure is 1 second
          multiplier: 1 # Failed waiting time multiple, next waiting time = multiplier * last interval
          max-attempts: 3 # max retries 
          stateless: true # true: stateless; False has status. If the transaction is included in the business, it is changed to false here
  • Enable local retry. If exceptions are always thrown during message processing, the request will not be sent to the queue, but will be retried locally by the consumer
  • When the maximum number of retries is reached, spring will return ack and the message will be discarded

four   The consumer module adds a failure policy (after the failure local retry function is enabled)

4.1 failure strategy

  • When local retry is enabled, the message is directly discarded after the maximum number of retries.
  • The three policies are inherited from the MessageRecovery interface
    • RejectAndDontRequeueRecoverer: after the retry is exhausted, reject directly and discard the message. This is the default
    • ImmediateRequeueMessageRecoverer: after the retry is exhausted, nack is returned and the message is re queued
    • RepublishMessageRecoverer: after the retry is exhausted, post the failure message to the specified switch

4.2 define the switch and queue for processing failure messages

  • No corresponding queue, switch and binding relationship will be automatically created, and nothing will be done
@Bean
public DirectExchange errorMessageExchange(){
    return new DirectExchange("error.direct");
}
@Bean
public Queue errorQueue(){
    return new Queue("error.queue", true);
}

// The routing key is key
@Bean
public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
}

4.3 add a failure policy component to the container

@Bean
public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    // error is the routing key
    return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
}

Tags: Java RabbitMQ Spring Boot

Posted on Fri, 24 Sep 2021 09:19:51 -0400 by CMC