1, Introduction to RabbitMQ
rabbitMQ official website: https://www.rabbitmq.com/
1. What is it
Reference article: https://zhuanlan.zhihu.com/p/157112243
Message queue is a container that holds messages during message transmission.
RabbitMQ is an open source message broker software (also known as message oriented middleware) that implements the advanced message queuing protocol (AMQP). RabbitMQ server is written in Erlang language, and clustering and failover are built on the framework of open telecommunications platform. All major programming languages have client libraries that communicate with proxy interfaces.
Corresponding versions of rabbitMQ and Erlang: https://www.rabbitmq.com/which-erlang.html
Rabbit Technology Co., Ltd. developed and supported rabbit MQ. At first, rabbit technology was a joint venture established by LSHIFT and CohesiveFT in 2007 and was acquired by SpringSource under VMware in April 2010. RabbitMQ became part of gopivot in May 2013.
RabbitMQ is a set of open source (MPL) Message Queuing service software. It is an open source implementation of Advanced Message Queuing Protocol (AMQP) provided by lsshift. It is written by Erlang, who is famous for its high performance, robustness and scalability.
2. What can I do
Message queue is a kind of technical service that can accept data, accept requests, store data, send data and so on.
It has the following functions:
-
Traffic peak shaving: for large concurrency, if you directly access the server, it is easy to cause server downtime. When we use MQ to accept requests for access, it will greatly alleviate the high concurrency. But this also brings a problem - the problem of slow access speed, but it is better than downtime.
-
Application decoupling: if we allow access between services instead of through MQ, this will bring a problem. For example, the order system will call the payment system, inventory system and logistics system. If there is a problem with the payment system, the whole business cannot be completed. If we use MQ, MQ will serve as a bridge between the order system and the payment system, inventory system and logistics system. When MQ finds something wrong with the payment system, MQ will supervise the payment system until it completes its task
-
Asynchronous processing: we may have A requirement that service A calls service B, but the execution of service B takes A certain time, but service A needs to know when the execution of service B is completed. Without MQ, we may need to call the callback function of service B within A certain time (ask service B if it is OK), but this method is not elegant at all. Using MQ can easily solve problems in.
For example, when you have a data to be migrated or too many concurrent requests, for example, you have 10W concurrent requests to place orders. Before these orders are warehoused, we can stack the order requests into the message queue to make them warehoused and executed stably and reliably.
3. What are the characteristics
Reliability: RabbitMQ uses some mechanisms to ensure reliability, such as persistence, transmission confirmation and release confirmation.
Flexible routing: messages are routed through the switch before they enter the queue. For typical routing functions, RabbitMQ has provided some built-in switches to implement. For more complex routing functions, multiple switches can be bound together, or their own switches can be implemented through plug-in mechanism.
Scalability: multiple RabbitMQ nodes can form a cluster, or dynamically expand the nodes in the cluster according to the actual business situation.
High availability: the queue can be mirrored on the machines in the cluster, so that the queue is still available in case of problems on some nodes.
Multiple protocols: RabbitMQ supports not only AMQP protocol, but also many message middleware protocols such as STOMP and MQTT
Multilingual client: RabbitMQ supports almost all common languages, such as Java, Python, Ruby, PHP, C#, JavaScript, etc.
Management interface: RabbitMQ provides an easy-to-use user interface, enabling users to monitor and manage messages, nodes in the cluster, etc.
Plug in mechanism: RabiMQ provides many plug-ins to extend from many aspects. Of course, you can also write your own plug-ins.
2, RabbitMQ installation
If you don't have the foundation of docker, you can read the articles in my docker column [Click to view]
1. Docker installation
(1)yum Update package to latest
yum update
(2)Install the required packages, yum-util provide yum-config-manager Function, the other two are devicemapper Drive dependent
yum install -y yum-utils device-mapper-persistent-data lvm2
(3)set up yum Source: alicloud
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
(4)install docker
yum install docker-ce -y
(5)View after installation docker edition
docker -v
(6) Install accelerated mirror
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://0wrdwnn6.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
2. Installing RabbitMQ in Docker
The traditional way is to pull the image first and run the container, but in this way, we also need to enter the container to set the account and password. (RebbitMQ defaults to guest, tourist role)
① Pull the image of rabbitMQ
docker pull rabbitmq:management
② Create and run containers
docker run -di --name=myrabbit -p 15672:15672 rabbitmq:management
③ Set the user name and password for rabbitmq
rabbitmqctl add_user admin(User name) admin(Password)
④ Set roles for users
rabbitmqctl set_user_tags admin administrator
⑤ Set permissions for users
rabbitmqctl set_permissions -p "/" admin ".\*" ".\*" ".\*" "/" Which one virtualhost admin Indicates which user to set ".\*" ".\*" ".\*" It respectively supports configuration, write and read
View current user list
rabbitmqctl list_users
We can set the account, password and role in one step
docker run -di --name myrabbit -e RABBITMQ_DEFAULT_USER=admin -e RABBITMQ_DEFAULT_PASS=admin -p 15672:15672 -p 5672:5672 -p 25672:25672 -p 61613:61613 -p 1883:1883 rabbitmq:management --hostname: Specifies the container host name --name: Specify container name -e RABBITMQ_DEFAULT_USER=admin: Set user name to admin -e RABBITMQ_DEFAULT_PASS=admin: Set password to admin -p: take mq Port numbers are mapped to local or run-time settings for users and passwords
We set the account, password and role in one step. After installing RabbitMQ in docker and using the above command.
We can use docker ps to view the process of docker container and find that rabbitMQ has been started successfully
We can use http: / / your IP address: 15672 to access the rabbit console (management plugin).
3, Roles in RabbitMQ
Role: 1: none
The management plugin cannot be accessed
Role 2: management: view your related node information
List the virtual machines that you can log in through AMQP
View the queues,exchanges and bindings information of your virtual machine node virtual hosts
View and close your channels and connections
View statistics about your virtual machine node virtual hosts. Include the activity information of other users in the virtual hosts of this node
Role 3: Policymaker
Include all management permissions
View, create and delete the policies and parameters information to which your virtual hosts belong
Role 4: Monitoring
Include all management permissions
List all virtual hosts, including those that cannot log in
View the connections and channels information of other users
View node level data, such as clustering and memory usage
View global statistics for all virtual hosts
Role 5: Administrator
Highest authority
You can create and delete virtual hosts
users can be viewed, created and deleted
View and create permissions
Close connections for all users
4, AMQP protocol
1. What is AMQP
AMQP, or Advanced Message Queuing Protocol, is an application layer standard Advanced Message Queuing Protocol that provides unified messaging services. It is an open standard of application layer protocol and is designed for message oriented middleware. The client and message middleware based on this protocol can deliver messages, which is not limited by different products and different development languages of the client / middleware. The implementations in Erlang include RabbitMQ and so on.
2. Why AMQP
Our goal is to implement a standard message middleware technology widely used in the whole industry, so as to reduce the overhead of enterprise and system integration, and provide industrial level integration services to the public.
Our aim is to make the capability of message oriented middleware finally possessed by the network itself through AMQP, and develop a series of useful applications through the wide use of message oriented middleware.
3. AMQP producer flow process
4. AMQP consumer circulation process
5, The core components of RabbitMQ
1. Principle of RabbitMQ
RabbitMQ has four cores: producer, exchange, queue and consumer
Core concept
Server: also known as Broker, it accepts client connections and implements AMQP entity services. Install rabbitmq server
Connection: connection, the network connection between the application and the Broker, TCP/IP / three handshakes and four waves
Channel: network channel. Almost all operations are carried out in the channel. The channel is the channel for message reading and writing. The client can establish access to each channel, and each channel represents a session task.
Message: Message: the data transmitted between the service and the application. It is composed of properties and body. Properties can modify the message, such as message priority, delay and other advanced features. Body is the content of the message body.
Virtual Host: virtual address, which is used for logical isolation and top-level message routing. A Virtual Host can have several exhanges and queues. There can be no Exchange with the same name in the same Virtual Host.
Exchange: the switch accepts messages and sends messages to the bound queue according to the routing key. If no exchange is specified in RabbitMQ, there will be a default exchange (default exchange, the default mode is direct mode) (it does not have the ability to store messages)
Bindings: virtual connection between Exchange and Queue. Multiple routing key s can be protected in binding.
Routing key: a routing rule that virtual machines can use to determine how to route a specific message.
Queue: queue: also known as Message Queue, Message Queue, saves messages and forwards them to consumers.
2. RabbitMQ overall architecture
3. Running process of RabbitMQ
6, Hello World of RabbitMQ
After all that has been said above, let's now demonstrate (simple mode) a rabbitMQ hello world.
1. java Native dependency
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.10.0</version>
</dependency>
2. Write producer
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Producer {
public static void main(String[] args) throws Exception{
// Create connection factory
ConnectionFactory factory = new ConnectionFactory();
// Set connection properties
factory.setHost("192.168.174.135");
factory.setPort(5672);
factory.setVirtualHost("/");
factory.setUsername("admin");
factory.setPassword("admin");
final String QUEUE_NAME = "queue01";
// Create connection
Connection producer = factory.newConnection();
// Add a channel
Channel channel = producer.createChannel();
/**
* queueDeclare : Declaration queue
* @param queue Queue name
* @param durable Do you want to persist the message locally
* @param exclusive Whether to enable exclusive mode (messages are shared or not)
* @param autoDelete Is the queue automatically deleted after the last queue consumes messages
* @param arguments Some other parameters of the queue
*/
channel.queueDeclare(QUEUE_NAME,false,false,false,null);
String message = "hello studious tiger!";
/**
* basicPublish : Simple release message
* @param exchange Switch used for message publishing (with default switch)
* @param routingKey Routing keyword (we don't have it now. We'll talk about it later. Now just write the queue name directly)
* @param mandatory true if the 'mandatory' flag is to be set
* @param props Some other parameters of the message, which we don't have now
* @param body Message body (byte type)
*/
channel.basicPublish("",QUEUE_NAME,null,message.getBytes("UTF-8"));
System.out.println("Message sent successfully!");
// close resource
if (channel != null && channel.isOpen()) {
try {
channel.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
if (connection != null) {
try {
connection.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
implement
It can be found that queue01 exists and the ready message is 1, indicating that the message is published successfully
3. Write consumer
import com.rabbitmq.client.*;
public class Customer {
public static void main(String[] args) throws Exception{
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.174.135");
factory.setPort(5672);
factory.setVirtualHost("/");
factory.setUsername("admin");
factory.setPassword("admin");
final String QUEUE_NAME = "queue01";
Connection producer = factory.newConnection();
Channel channel = producer.createChannel();
/**
* basicConsume : Simple consumer news
* @param queue Queue name
* @param autoAck autoAck When set to true, the message queue can ignore whether the message consumer has finished processing the message,
* All messages are sent all the time. However, in fair distribution, that is, if autoAck is set to false, it will be confiscated after sending a message
* Until the message consumer successfully consumes the information receipt of the message, it will not continue to send messages to the message.
* @param deliverCallback This callback function is executed when a message is received
* @param cancelCallback This callback function is executed when the consumer is cancelled
*/
// Ramdas expression (DeliverCallback is a functional interface. After we use Ramdas expression, we don't need to use implementation class)
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println( new String(message.getBody(),"UTF-8"));
};
// Ramdas expression
CancelCallback cancelCallback = consumerTag ->{
System.out.println("Message interrupt...");
};
channel.basicConsume(QUEUE_NAME,true,deliverCallback,cancelCallback);
// close resource
if (channel != null && channel.isOpen()) {
try {
channel.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
if (connection != null) {
try {
connection.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
Note: it is assumed that the 8 - Lambdas (Ramdas) expression is used above. You should ensure that the environment is 1.8
implement
It can be found that the ready message is 0, indicating that the consumption is successful
3. Extraction tool class
We can find that it is troublesome to repeatedly establish channels and close resources every time. We can extract them into tool classes
public class RabbitMQUtils {
/**
* @return Return channel
* @throws Exception
*/
public static Channel getChannel() throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.174.135");
factory.setPort(5672);
factory.setVirtualHost("/");
factory.setUsername("admin");
factory.setPassword("admin");
Connection producer = factory.newConnection();
return producer.createChannel();
}
/**
* close resource
* @param channel channel
* @param connection connect
*/
public static void closeSource(Channel channel,Connection connection) {
if (channel != null && channel.isOpen()) {
try {
channel.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
if (connection != null) {
try {
connection.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
7, Message response [simple mode work]
1. Introduction
The book continues from the above. In the above, we set autoAck to true, but this method is actually unsafe, because when the consumer receives the message from the queue, it will immediately send a reply message to the queue, and the queue will delete the message. This will lead to a problem: when the consumer accepts the message, it does not complete its corresponding task, and because the message has been deleted in the queue In addition, the task of consumers will completely fail!
So we usually use manual response
channel.basicAck(long deliveryTag, boolean multiple): indicates a response
Channel. Basicnack (long deliverytag, Boolean multiple, Boolean request): indicates a rejection response
Channel. Basicreject (long deliverytag, Boolean request): indicates a rejection response
Differences between basicNack and basicReject:
When the request is true, it means that the message is re queued and the message will be pushed again.
We can also see from the parameters that there is one more boolean multiple in basicNack, and multiple means rejecting multiple responses at one time (for example, basicNack(5, true, true) means rejecting 1, 2, 3, 4 and 5 messages and re queuing the messages; basicNack(5, true, true) means rejecting the fifth message and re queuing the messages)
Schematic diagram of message queue re reply mechanism:
2. Code implementation
Next, we do an experiment: we need a producer and two consumers (consumer 1 processes business for 1 second and consumer 2 processes business for 30 seconds). The producer is used to send messages and the consumer is used to receive messages. Because exchange uses the polling mechanism by default, when the producer produces aa, bb, cc and dd messages, aa and cc should be accepted by one consumer and bb and dd by another consumer.
When consumer 2 receives the second message, we artificially create downtime, which makes consumer 2 unable to continue to accept the message. Let's verify whether the message that should have been given to consumer 2 will be sent to consumer 1 when consumer 2 has no response queue.
Producer code
public class Producer {
public static final String ACK_QUEUE = "ack_queue";
public static void main(String[] args) throws Exception{
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(ACK_QUEUE,false,false,false,null);
Scanner scanner = new Scanner(System.in);
while (scanner.hasNext()){
String message = scanner.next();
channel.basicPublish("",ACK_QUEUE,null,message.getBytes("UTF-8"));
System.out.println("Message sent successfully:"+message);
}
RabbitMQUtils.closeSource(channel,connection);
}
}
Consumer 01 code
public class Customer01 {
public static final String ACK_QUEUE = "ack_queue";
public static void main(String[] args) throws Exception {
System.out.println("Customer01 Waiting to accept messages in the message queue...");
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
DeliverCallback deliverCallback = (consumerTag, message)->{
// Sleep for 1 second
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println( new String(message.getBody(),"UTF-8"));
// Manual answer
channel.basicAck(message.getEnvelope().getDeliveryTag(),false);
};
CancelCallback cancelCallback = consumerTag ->{
System.out.println("Message interrupt...");
};
// No auto answer
Boolean autoAck = false;
channel.basicConsume(ACK_QUEUE,autoAck,deliverCallback,cancelCallback);
RabbitMQUtils.closeSource(channel,connection);
}
}
Consumer 02 code
public class Customer02 {
public static final String ACK_QUEUE = "ack_queue";
public static void main(String[] args) throws Exception {
System.out.println("Customer02 Waiting to accept messages in the message queue...");
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
DeliverCallback deliverCallback = (consumerTag, message)->{
// Sleep for 10 seconds
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println( new String(message.getBody(),"UTF-8"));
// Manual answer
channel.basicAck(message.getEnvelope().getDeliveryTag(),false);
};
CancelCallback cancelCallback = consumerTag ->{
System.out.println("Message interrupt...");
};
// No auto answer
Boolean autoAck = false;
channel.basicConsume(ACK_QUEUE,autoAck,deliverCallback,cancelCallback);
RabbitMQUtils.closeSource(channel,connection);
}
}
When the producer sends dd, I turn off consumer 2. The implementation result shows that dd has been received and processed by consumer 1 (dd should have been received by consumer 2). It shows that the theory in vii-1 is correct.
3. Unfair distribution strategy
rabbitmq uses the polling distribution policy by default, which has an obvious disadvantage - it can't achieve "more work for those who can". Therefore, in general, we use unfair distribution strategies more often.
To realize non publication distribution, we only need to set channel.basicQos(1) on the consumer side, and channel.basicQos(0) is the default polling distribution.
Based on the code of seven - 2, those who can do more work
4. Pre value
We can know from the previous study that the messages in the queue are delivered to consumers through the channel. The so-called pre value is like setting the capacity of the channel. To use the pre value, you need to enable autoAck=false, and you need to use basiack to confirm the message
Official explanation
For cases when multiple consumers share a queue, it is useful to be able to specify how many messages each consumer can be sent at once before sending the next acknowledgement. This can be used as a simple load balancing technique or to improve throughput if messages tend to be published in batches. For example, if a producing application sends messages every minute because of the nature of the work it is doing.
Note that RabbitMQ only supports channel-level prefetch-count, not connection or size based prefetching.
When multiple users share a queue, you can specify how many messages each user can send at a time before sending the next confirmation. This can be used as a simple load balancing technique or to improve throughput when messages tend to be published in batches. For example, if a production application sends messages every minute because of the nature of the work it does.
Note that RabbitMQ only supports channel level prefetch counts and does not support connection or size based prefetching.
In unfair distribution, we set channel.basicQos(1). For pre value, we also implement it through channel.basicQos(count), but the count here is no longer 1 or 0, but other numbers (for example, we can use channel.basicQos(2) and channel.basicQos(5)).
The above figure is not absolute, it is just a great possibility. For example, the total number of messages is 10, and there is a voltammetry inside. Try to balance these 10 messages (load balancing) according to the setting of prefetchcount. The above figure is just an example to make you understand more vividly (not the true principle).
Based on the code of vii-2, the demonstration of pre value is realized
We will send 7 messages. If nothing unexpected happens, consumer 01 will accept 2 messages (1 second for business processing) and consumer 02 will accept 5 messages (30 seconds for business processing)
Consumer 01 sets the preset value to 2
Consumer 02 sets the preset value to 5
8, Message persistence
1. Introduction
We did not implement the persistence of queues and messages above, which is actually unwise, because when rabbitmq goes down due to an accident, the queues and messages in rabbitmq will no longer exist. Generally, non persistent message queues are rarely used in actual development, because this will greatly increase the risk of message loss. We can't guarantee that our rabbitmq is absolutely reliable. Because it is very necessary for the persistence of queues and messages in queues.
If you are impressed, when the producer declares the queue, we set three false to the method of declaring the queue (queueDeclare), the first of which means that the queue is not persistent. In other words, when we change the original false to true, we realize the persistence of the queue.
2. Queue persistence
No queue before persistence:
Queue after persistence: (after a queue is declared, its characteristics cannot be changed unless it is deleted and redeclared)
Delete original queue
Persistence is implemented when the queue is declared on the producer side
3. Message persistence
The message is also declared in the producer. If you are impressed, the third parameter (BasicProperties props) in the message publishing method (basicPublish) is set to null. We only need to use messageproperties.persistent_ TEXT_ Plan replaces null to achieve message persistence.
4. Release confirmation [simple mode work]
We have enabled message persistence above, but there is still a case that when the producer delivers messages to the queue through the switch, if the message delivery fails during this period, it will also affect the robustness of the service. We need a mechanism: the producer can know whether the message is published successfully (when the message fails to be published, it can be personalized)
Open release confirmation
Producers can use channel.confirmSelect() to start publishing confirmation and channel.waitForConfirms() to receive publishing confirmation.
③ Single release confirmation
The so-called single release confirmation means that every time a producer publishes a message to the queue, it will wait for the queue release confirmation. In this way, we can clearly know whether each release is successful or not, but the problem brought by this mode is that it affects efficiency
code:
/**
* Single confirmation release
* @throws Exception
*/
public static void publishMessageSingle() throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Queue name
String queueName= UUID.randomUUID() + "";
// Declaration queue
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation (don't forget * * *)
channel.confirmSelect();
// start time
long begin = System.currentTimeMillis();
int messageCounts = 1000;
// Send 1000 messages
for (int i = 0; i < messageCounts; i++) {
String message = i+1+"";
channel.basicPublish("",queueName,null,message.getBytes("UTF-8"));
// Single acknowledgement to queue
boolean tags = channel.waitForConfirms();
// Judge whether the transmission is successful
if (tags){
System.out.println("The first" + message + " Messages sent successfully...");
}else {
System.out.println("The first" + message + " Messages succeeded or failed...");
}
}
// End time
long end = System.currentTimeMillis();
System.out.println("[publishMessageSingle time consuming "+(end-begin)+" ms]");
}
The time to complete publishing is 1123 milliseconds
② Batch release confirmation
The so-called batch publish confirmation is to set a batch value. When the number of published messages meets the batch value each time, publish confirmation is performed. Compared with the single release confirmation mode, this mode greatly improves the efficiency, but the problem is that it can not ensure the success of each release.
code:
/**
* Batch confirmation release
* @throws Exception
*/
public static void publishMessageBatch() throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Queue name
String queueName= UUID.randomUUID() + "";
// Declaration queue
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation (don't forget to open * * *)
channel.confirmSelect();
// start time
long begin = System.currentTimeMillis();
int messageCounts = 1000;
// Set the batch value to 100
int batchSize = 100;
// Send 1000 messages
for (int i = 0; i < messageCounts; i++) {
int message = i+1;
channel.basicPublish("",queueName,null,(message+"").getBytes("UTF-8"));
// Batch confirmation (batch is 100)
if (message%batchSize==0){
// confirm
boolean tags = channel.waitForConfirms();
// Judge whether the transmission is successful
if (tags){
System.out.println("The first" + message/100 + " Batch message sent successfully...");
}else {
System.out.println("The first" + message/100 + " Batch message success failure...");
}
}
}
// End time
long end = System.currentTimeMillis();
System.out.println("[publishMessageBatch time consuming "+(end-begin)+" ms]");
}
The time to complete the release is 64 milliseconds, which is significantly more efficient than the confirmation of a single release
③ Asynchronous publish confirmation
The so-called asynchronous publishing is that we encapsulate messages into a map like data structure (rabbitmq encapsulates itself). The advantage is that each message can have a deliveryTag. When publishing, we publish all messages at once. It is asynchronous for publishing confirmation. It is detected that a message publishing failure is detected in the asynchronous program, The message number can be returned to the producer so that the producer can accurately know which release failed. Asynchronous release confirmation is the most complex mode, but this mode is the most cost-effective. In this mode, you can clearly know whether each release is successful while ensuring efficiency.
rabbitmq provides a release confirmation listener channel.addConfirmListener(ackCallback,nackCallback) to channel, which can realize asynchronous release confirmation. We need to write two callback functions, ackCallback (the callback function called when publishing succeeds) and nackCallback (the callback function called when publishing fails) through the Ramdas expression.
code:
/**
* Asynchronous confirmation Publishing
* @throws Exception
*/
public static void publishMessageAsync() throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Queue name
String queueName = UUID.randomUUID() + "";
// Declaration queue
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation
channel.confirmSelect();
// start time
long begin = System.currentTimeMillis();
int messageCounts = 1000;
// Callback function called when publishing is successful
// deliveryTag: the tag of the message
// multiple: batch confirmation
com.rabbitmq.client.ConfirmCallback ackCallback = (deliveryTag,multiple)->{
System.out.println("news " + deliveryTag + " Published successfully...");
};
// Callback function called when publishing fails
ConfirmCallback nackCallback = (deliveryTag,multiple)->{
System.out.println("news " + deliveryTag + " Publishing failed...");
};
/**
* Set up a publish confirmation listener
* ackCallback : Listening when publishing is successful
* nackCallback : Listening when publishing fails
*/
channel.addConfirmListener(ackCallback,nackCallback);
// Send 1000 messages
for (int i = 0; i < messageCounts; i++) {
int message = i+1;
channel.basicPublish("",queueName,null,(message+"").getBytes("UTF-8"));
}
long end = System.currentTimeMillis();
System.out.println("publishMessageAsyn time consuming "+(end-begin)+" ms");
}
Discovery is indeed asynchronous and can ensure efficiency.
Question: how do I handle asynchronous unacknowledged messages?
The best paradigm is to store the confirmed messages in a memory based queue that can be accessed by the publishing thread. For example, use a queue similar to concurrent linked queue to communicate between confirm callbacks and the publishing process.
code:
/**
* Asynchronous confirmation Publishing
* @throws Exception
*/
public static void publishMessageAsync() throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Queue name
String queueName = UUID.randomUUID() + "";
// Declaration queue
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation
channel.confirmSelect();
/**
* [1]A thread safe and orderly hash table, used with high concurrency, is used to store published messages
* 1. Easily associate sequence numbers with messages
* 2. You can delete entries in batches according to the serial number
* 3. Support high concurrency (multithreading)
*/
ConcurrentSkipListMap<Long, String> messageMap = new ConcurrentSkipListMap<>();
// Callback function called when publishing is successful
// deliveryTag: the tag of the message
// multiple: batch confirmation
com.rabbitmq.client.ConfirmCallback ackCallback = (deliveryTag,multiple)->{
// [3.1] how to batch delete
if (multiple){
// The headMap(K toKey) method is used to return a partial view where the key of this map is strictly smaller than the toKey.
// In fact, it is to obtain the content before deliveryTag
ConcurrentNavigableMap<Long, String> map = messageMap.headMap(deliveryTag);
map.clear();
}else {
// If it is not a batch, it can be deleted directly
messageMap.remove(deliveryTag);
}
System.out.println("news " + deliveryTag + " Published successfully...");
};
// Callback function called when publishing fails
ConfirmCallback nackCallback = (deliveryTag,multiple)->{
// [3.2]
String message = messageMap.get(deliveryTag);
System.out.println("news: " + message + "[Serial number:" + deliveryTag + "] ,Publishing failed...");
};
/**
* Set up a publish confirmation listener
* ackCallback : Listening when publishing is successful
* nackCallback : Listening when publishing fails
*/
channel.addConfirmListener(ackCallback,nackCallback);
// start time
long begin = System.currentTimeMillis();
int messageCounts = 1000;
// Send 1000 messages
for (int i = 0; i < messageCounts; i++) {
int message = i+1;
channel.basicPublish("",queueName,null,(message+"").getBytes("UTF-8"));
// [2] Record the sent message in the map. channel.getNextPublishSeqNo() indicates the sequence number of the next sent message
messageMap.put(channel.getNextPublishSeqNo(),message+"");
}
long end = System.currentTimeMillis();
System.out.println("publishMessageAsyn time consuming "+(end-begin)+" ms");
}
9, Commonly used modes of RabbitMQ (!!!)
1. java Native
java Native dependency
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.10.0</version>
</dependency>
It is worth noting that in the program, we can define switches on both the producer side and the consumer side. However, the only queue can only be defined at the early consumer end, because one consumer corresponds to one queue. However, I still suggest that the declaration of the switch and the declaration of the queue can be put together, because the message published by the producer is sent to the queue through the switch. If the declaration of your switch and queue do not exist at the same time, it will be lost when sending or receiving the first message.
① Fanout mode
In publish and subscribe mode, when multiple queues bin d to the same exchange, when the exchange forwards messages, all queues bound to it will receive messages. Then each queue corresponds to a consumer, so that the message released by a producer can be received by multiple consumers at the same time.
producer:
public class Producer {
public static final String EXCHANGE_NAME = "myExchange_fanout";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare switch (name and type)
channel.exchangeDeclare(EXCHANGE_NAME,BuiltinExchangeType.FANOUT);
// Release news
Scanner scanner = new Scanner(System.in);
while (scanner.hasNext()){
String message = scanner.next();
channel.basicPublish(EXCHANGE_NAME,"",null,message.getBytes("UTF-8"));
System.out.println("[Producer]Message published successfully:"+message);
}
}
}
Consumer 01:
public class Customer01 {
public static final String EXCHANGE_NAME = "myExchange_fanout";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare the queue (this is a temporary queue, and the queue name is random)
String queue = channel.queueDeclare().getQueue();
// Binding switch
channel.queueBind(queue,EXCHANGE_NAME,"");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer01]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption news
channel.basicConsume(queue,true,deliverCallback,cancelCallback);
}
}
Consumer 02:
public class Customer02 {
public static final String EXCHANGE_NAME = "myExchange_fanout";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare the queue (this is a temporary queue with random queue name. When the message is consumed, the queue will be deleted automatically)
String queue = channel.queueDeclare().getQueue();
// Binding switch
channel.queueBind(queue,EXCHANGE_NAME,"");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer02]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer02]Failed to receive message...");
};
// Consumption news
channel.basicConsume(queue,true,deliverCallback,cancelCallback);
}
}
② Direct mode
The difference between direct mode and fanout mode is that a routing key (you can think of it as routing, label, classification...) is added. Through this restriction, even if multiple queues are bound to the same exchange, we can specify the exchange when forwarding messages, Forwards the message to the specified queue. [Note: multiple routing keys can be added to the same queue]
Next, we will implement the following model. The routing key of consumer 01 is info and warning. The routing key of consumer 02 is error.
producer:
public class Producer {
public static final String EXCHANGE_NAME = "myExchange_direct";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare switch (name and type)
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
// Release news
Scanner scanner = new Scanner(System.in);
while (scanner.hasNext()){
String messageAll = scanner.next();
// Cut the input content according to "-" to get message (info[0]) and routing king (info[1])
String[] info = messageAll.split("-");
channel.basicPublish(EXCHANGE_NAME,info[1],null,info[0].getBytes("UTF-8"));
System.out.println("[Producer]Message published successfully:"+info[0]);
}
}
}
Consumer 01 (routing key is info and warning):
public class Customer01 {
public static final String EXCHANGE_NAME = "myExchange_direct";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declaration queue
channel.queueDeclare("queue_direct01",false,false,false,null);
// Binding switch (routing key is info and warning)
channel.queueBind("queue_direct01",EXCHANGE_NAME,"info");
channel.queueBind("queue_direct01",EXCHANGE_NAME,"warning");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer01]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption news
channel.basicConsume("queue_direct01",true,deliverCallback,cancelCallback);
}
}
Consumer 02 (routing key is error):
public class Customer02 {
public static final String EXCHANGE_NAME = "myExchange_direct";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declaration queue
channel.queueDeclare("queue_direct02",false,false,false,null);
// Binding switch (routing key is error)
channel.queueBind("queue_direct02",EXCHANGE_NAME,"error");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer02]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer02]Failed to receive message...");
};
// Consumption news
channel.basicConsume("queue_direct02",true,deliverCallback,cancelCallback);
}
}
③ Topic mode
Topic mode. From the schematic diagram, we can find that the difference between topic mode and Direct mode is that topic mode can be fuzzy matched. How to understand fuzzy matching? In fact, it is similar to regular expressions (but simpler than regular expressions).
The routing key in topic mode can be used for fuzzy matching, such as *. orange.* *.*.rabbit lazy#
.: indicates the boundary of level
*: indicates that level 1 can be vaguely matched (representing only one word)
#: indicates that it can be vaguely assigned with 0 or multiple levels (it can represent multiple words or not)
Case:
① If the restriction adjustment added by the switch when forwarding messages is lazy.xxxx.xxxx.xxxx, only the queue with the restriction condition (routing key) of lazy. # can receive messages, because # indicates that it can match level 0 or multiple levels.
② If the restriction adjustment added by the switch when forwarding messages is lazy.orange.xxxx, only queues with routing key of lazy. # and *. Orange. * can receive messages.
③ If the restriction adjustment added by the switch when forwarding messages is lazy.orange.xxxx.xxxx, only queues with routing key of lazy. # can receive messages. The reason why the *. Orange. * queue cannot receive messages is that there are only 1 levels under orange. Obviously, there are 2 levels under orange in lazy.orange.xxxx.xxxx.
④ If the restriction adjustment added by the switch when forwarding messages is lazy.orange.rabbit, only queues with routing key of lazy. #, *. Orange. * and *. *. Rabbit can receive messages.
Next, we implement the code according to the above case
producer:
public class Producer {
public static final String EXCHANGE_NAME = "myExchange_topic";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare switch (name and type)
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC);
// Encapsulate routes and messages into map s
HashMap<String, String> routingkeyMessageMap = new HashMap<>();
routingkeyMessageMap.put("lazy.xxxx.xxxx.xxxx","This message will be Customer02 receive");
routingkeyMessageMap.put("lazy.orange.xxxx","This message will be Customer01 and Customer02 receive");
routingkeyMessageMap.put("lazy.orange.xxxx.xxxx","This message will be Customer02 receive");
routingkeyMessageMap.put("lazy.orange.rabbit","This message will be Customer01 and Customer02 Receive receive");
for (Map.Entry<String, String> map : routingkeyMessageMap.entrySet()) {
String routingKey = map.getKey();
String message = map.getValue();
// Release news
channel.basicPublish(EXCHANGE_NAME,routingKey,null,message.getBytes("UTF-8"));
System.out.println("[Producer]Message published successfully:"+message);
}
}
}
Consumer 01: (routing key is. orange.)
public class Customer01 {
public static final String EXCHANGE_NAME = "myExchange_topic";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declaration queue
channel.queueDeclare("queue_topic01",false,false,false,null);
// Binding switch (routing key is *. orange. *)
channel.queueBind("queue_topic01",EXCHANGE_NAME,"*.orange.*");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer01]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption news
channel.basicConsume("queue_topic01",true,deliverCallback,cancelCallback);
}
}
Consumer 02: (routing key is.. rabbit and lazy. #)
public class Customer02 {
public static final String EXCHANGE_NAME = "myExchange_topic";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declaration queue
channel.queueDeclare("queue_topic02",false,false,false,null);
// Binding switch (routing key is *. *. rabbit and lazy. #)
channel.queueBind("queue_topic02",EXCHANGE_NAME,"*.*.rabbit");
channel.queueBind("queue_topic02",EXCHANGE_NAME,"lazy.#");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer02]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer02]Failed to receive message...");
};
// Consumption news
channel.basicConsume("queue_topic02",true,deliverCallback,cancelCallback);
}
}
④ Work mode
When there are a large number of messages, we want multiple worker threads to execute messages at the same time. In order to avoid repeated work, each message can only be processed once and cannot be processed multiple times, which leads to two distribution mechanisms - polling mode and fair distribution.
Polling mode (implemented in [7] and [8])
Fair distribution (achieved in [VII] and [viii])
⑤ Header mode
The header mode is also relatively simple. When the exchange sends a message, it needs to determine which queue the message is sent to through parameters. For example, if the header in queue 1 (queue01) is x=1 and the header in queue 2 (queue02) is y=1, only queue 1 (queue01) will receive messages if the condition of the exchange when sending messages is x=1.
10, Dead letter queue and delay queue
1. Dead letter queue
What is a dead letter queue? Dead letter queue generally speaking, the producer delivers messages to the queue, and the consumer takes out messages from the queue for consumption, but sometimes some messages in the queue cannot be consumed due to specific reasons. If there is no subsequent processing, such messages will become dead letters, and all dead letters will be placed in the dead letter queue.
Sources of dead letter queue: Message TTL (Time to Liv lifetime) expires, the queue reaches the maximum length, the message is rejected (basic.region or basic.nack) and request = false, and manual response is enabled (autoAck is false).
Architecture diagram
key of argument
① Message forwarding dead letter queue caused by message timeout
The following shows the message forwarding dead letter queue caused by message timeout (TTL)
We create customer01 to implement the two switches, two queues and binding relationship in the above architecture diagram
public class Customer01 {
public static final String NORMAL_EXCHANGE = "normal_exchange";
public static final String DEAD_EXCHANGE = "dead_exchange";
public static final String NORMAL_QUEUE = "normal_queue";
public static final String DEAD_QUEUE = "dead_queue";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare normal switch and dead letter switch
channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
// Declare normal queue
Map<String, Object> arguments = new HashMap<>();
arguments.put("x-dead-letter-exchange",DEAD_EXCHANGE); // Set the dead letter switch corresponding to the normal queue
arguments.put("x-dead-letter-routing-key","tiger"); // Set the routing key of the dead letter switch
//arguments.put("x-message-ttl",10000); // Set the expiration time of the message (unit: ms)
channel.queueDeclare(NORMAL_QUEUE,false,false,false,arguments);
// Declare dead letter queue
channel.queueDeclare(DEAD_QUEUE,false,false,false,null);
// Bind a normal switch to a normal queue
channel.queueBind(NORMAL_QUEUE,NORMAL_EXCHANGE,"studious");
// Bind the switch of dead letter to the queue of dead letter
channel.queueBind(DEAD_QUEUE,DEAD_EXCHANGE,"tiger");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer01]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption news
channel.basicConsume(NORMAL_QUEUE,true,deliverCallback,cancelCallback);
}
}
Create a consumer (producer):
public class Producer {
public static final String NORMAL_EXCHANGE = "normal_exchange";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Set the expiration time of the message (unit: ms) Expiration:
AMQP.BasicProperties props = new AMQP.BasicProperties().builder().expiration("10000").build();
// Release news
for (int i = 0; i <10 ; i++) {
String message = "news"+(i+1);
channel.basicPublish(NORMAL_EXCHANGE,"studious",props,message.getBytes("UTF-8"));
System.out.println("[Producer]Message published successfully:"+message);
}
}
}
Next, let's run customer01 to implement two switches, two queues and binding relationship, and then close customer01. Then run producer to send a message to the queue. Because no consumer accepts it, the message expires after 10 seconds. It was originally in normal_ The 10 messages in the queue will be forwarded to dead_ Go to the queue. The demonstration is as follows:
Create customer01:
public class Customer02 {
public static final String DEAD_QUEUE = "normal_queue";
public static void main(String[] args) throws Exception {
// Establish a connection and turn on the channel
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer02]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer02]Failed to receive message...");
};
// Consumption news
channel.basicConsume(DEAD_QUEUE,true,deliverCallback,cancelCallback);
}
}
In fact, it's over here. The purpose of our experiment is to test the dead letter queue.
② Message forwarding dead letter queue due to full queue
② The following shows the message forwarding dead letter queue caused by the full queue
We need to set the maximum length of a queue in the normal queue (note that we do not need to set the expiration time of the queue). In this way, when the normal queue is full, redundant messages will enter the dead letter queue.
Producer code (no need to set expiration time)
public class Producer {
public static final String NORMAL_EXCHANGE = "normal_exchange";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Release news
for (int i = 0; i <10 ; i++) {
String message = "news"+(i+1);
channel.basicPublish(NORMAL_EXCHANGE,"studious",null,message.getBytes("UTF-8"));
System.out.println("[Producer]Message published successfully:"+message);
}
}
}
Consumer 01 code (set the maximum length of the queue):
This is the only difference from customer01 in ①.
public class Customer01 {
public static final String NORMAL_EXCHANGE = "normal_exchange";
public static final String DEAD_EXCHANGE = "dead_exchange";
public static final String NORMAL_QUEUE = "normal_queue";
public static final String DEAD_QUEUE = "dead_queue";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare normal switch and dead letter switch
channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
// Declare normal queue
Map<String, Object> arguments = new HashMap<>();
arguments.put("x-dead-letter-exchange",DEAD_EXCHANGE); // Set up dead letter switch
arguments.put("x-dead-letter-routing-key","tiger"); // Set the routing key of the dead letter switch
//----------------------Sets the maximum length of the queue-----------------------//
arguments.put("x-max-length",6); // Set the maximum length of a normal queue to 6
//-------------------------------------------------------------//
channel.queueDeclare(NORMAL_QUEUE,false,false,false,arguments);
// Declare dead letter queue
channel.queueDeclare(DEAD_QUEUE,false,false,false,null);
// Bind a normal switch to a normal queue
channel.queueBind(NORMAL_QUEUE,NORMAL_EXCHANGE,"studious");
// Bind the switch of dead letter to the queue of dead letter
channel.queueBind(DEAD_QUEUE,DEAD_EXCHANGE,"tiger");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
System.out.println("[Customer01]Message received successfully:"+ new String(message.getBody(),"UTF-8"));
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption news
channel.basicConsume(NORMAL_QUEUE,true,deliverCallback,cancelCallback);
}
}
We delete the original queue, and then start customer01 to rebuild two switches, two queues and binding relationships. Then close costomer01 (if costomer01 is always on, the messages published by the producer will be consumed directly, which is meaningless), and start the producer to publish 10 messages (because the maximum length of the normal queue is 6, four messages will enter the dead letter queue).
③ Message forwarding dead letter queue caused by message rejection
③ The following shows the message forwarding dead letter queue caused by message rejection
Set automatic response in the consumer (autoAck: false); Reject (channel.basicReject); Do not re queue (request: false).
Consumer code (set manual response, set refuse to receive, and don't put it back in the queue)
public class Customer01 {
public static final String NORMAL_EXCHANGE = "normal_exchange";
public static final String DEAD_EXCHANGE = "dead_exchange";
public static final String NORMAL_QUEUE = "normal_queue";
public static final String DEAD_QUEUE = "dead_queue";
public static void main(String[] args) throws Exception {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
// Declare normal switch and dead letter switch
channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
// Declare normal queue
Map<String, Object> arguments = new HashMap<>();
arguments.put("x-dead-letter-exchange",DEAD_EXCHANGE); // Set up dead letter switch
arguments.put("x-dead-letter-routing-key","tiger"); // Set the routing key of the dead letter switch
//arguments.put("x-max-length",6); // Set the maximum length of a normal queue to 6
//arguments.put("x-message-ttl",10000); // Set the expiration time of the message (unit: ms)
channel.queueDeclare(NORMAL_QUEUE,false,false,false,arguments);
// Declare dead letter queue
channel.queueDeclare(DEAD_QUEUE,false,false,false,null);
// Bind a normal switch to a normal queue
channel.queueBind(NORMAL_QUEUE,NORMAL_EXCHANGE,"studious");
// Bind the switch of dead letter to the queue of dead letter
channel.queueBind(DEAD_QUEUE,DEAD_EXCHANGE,"tiger");
// Receive successful callback function
DeliverCallback deliverCallback = (consumerTag, message)->{
String msg = new String(message.getBody(), "UTF-8");
if ("Message 8".equals(msg)){
// [reject]: channel.basicReject; Don't put it back in the queue: request: false
channel.basicReject(message.getEnvelope().getDeliveryTag(),false);
System.out.println("[Customer01]Receiving message [failed]:"+ new String(message.getBody(),"UTF-8"));
}else{
System.out.println("[Customer01]Received message [successful]:"+ new String(message.getBody(),"UTF-8"));
}
};
// Receive failed callback function
CancelCallback cancelCallback = consumerTag ->{
System.out.println("[Customer01]Failed to receive message...");
};
// Consumption message ([set manual response] autoAck: false)
channel.basicConsume(NORMAL_QUEUE,false,deliverCallback,cancelCallback);
}
}
We delete the original queue, and then start customer01 to rebuild two switches, two queues and binding relationships. Then start the producer to publish 10 messages (because consumer 01 refuses to receive "message 8", one message will enter the dead letter queue, and the remaining 9 messages will be consumed successfully).
2. Delay queue
The so-called delay queue is the queue where the expiration time message is set (for example, the above normal_queue queue queue). We can understand it from two aspects, because through the above learning, I can know that there are two ways to set the expiration time of messages:
-
First, when creating a queue, set all messages in the whole queue to delay the specified time. This method is not flexible, but it also has usage scenarios.
-
Second, when sending messages, set the expiration time for messages. When multiple producers publish messages through a switch, messages with different expiration times in a queue can be realized, which is more flexible.
Usage scenario of delay queue:
- If the order is not paid after 10 minutes, it will be automatically cancelled.
- If a new store is created and no goods are published within 10 days, a message reminder will be sent automatically.
- After new users register, if they do not log in within 3 days, they will be reminded by SMS.
- The user initiates a refund. If it is not processed after 3 days, the administrator will be notified for processing.
- To book a meeting, participants need to be notified 10 minutes in advance.
We will implement delay queue in [Xi]
11, Integrating springboot
springboot dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
1. Implement the delay queue in [X. 2] (set the delay time of the queue)
It is not flexible to declare that the expiration time of the whole queue is the same, so that all messages have the same delay time as long as they enter the queue
Model
In the following model, we can find that there are two switches (exchange_X, exchange_Y), three queues (queue_a, queue_b, queue_d), a producer and a customer. Where queue_ a,queue_ B belongs to the delay queue (the delay time is 10s and 20s respectively)_ D is the dead letter queue
Step 1: configure the connection profile
# Service port
server:
port: 8080
# Configure rabbitmq service
spring:
rabbitmq:
username: admin
password: admin
virtual-host: /
host: 192.168.174.136
port: 5672
Step 2: declare the switch
Don't lead the wrong package, huh
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Date;
@Configuration
public class rabbitmqConfig {
// Declare switch name
public static final String EXCHANGE_X = "exchange_X";
public static final String DEAD_LETTER_EXCHANGE_Y = "exchange_Y";
// Declaration queue name
public static final String QUEUE_A = "queue_a";
public static final String QUEUE_B = "queue_b";
public static final String DEAD_LETTER_QUEUE_D = "queue_d";
/**
* Declare exchange_X
* The first false indicates no persistence
* The second false indicates that it is not automatically deleted
* @return
*/
@Bean("exchange_X")
public DirectExchange exchange_X(){
return new DirectExchange(EXCHANGE_X,false,false);
}
/**
* Declare dead letter exchange_Y
* The first false indicates no persistence
* The second false indicates that it is not automatically deleted
* @return
*/
@Bean("exchange_Y")
public DirectExchange exchange_Y(){
return new DirectExchange(DEAD_LETTER_EXCHANGE_Y,false,false);
}
Step 3: declare the queue
/**
* Claim delay queue_a
* x-dead-letter-routing-key The routing key of the dead letter switch bound to the dead letter queue shall be consistent with that of the dead letter switch. The message cannot be received by the dead letter queue
* @return
*/
@Bean("queue_a")
public Queue queue_a(){
Map<String, Object> arguments = new HashMap<>();
// Set up dead letter switch
arguments.put("x-dead-letter-exchange",DEAD_LETTER_EXCHANGE_Y);
// Set dead letter routing key = dead
arguments.put("x-dead-letter-routing-key","dead");
// Set TTL expiration time
arguments.put("x-message-ttl",10000);
return QueueBuilder.nonDurable(QUEUE_A).withArguments(arguments).build();
}
/**
* Claim delay queue_b
* x-dead-letter-routing-key The routing key of the dead letter switch bound to the dead letter queue shall be consistent with that of the dead letter switch. The message cannot be received by the dead letter queue
* @return
*/
@Bean("queue_b")
public Queue queue_b(){
Map<String, Object> arguments = new HashMap<>();
// Set up dead letter switch
arguments.put("x-dead-letter-exchange",DEAD_LETTER_EXCHANGE_Y);
// Set dead letter routing key = dead
arguments.put("x-dead-letter-routing-key","dead");
// Set TTL expiration time
arguments.put("x-message-ttl",20000);
return QueueBuilder.nonDurable(QUEUE_B).withArguments(arguments).build();
}
/**
* Declare dead letter queue_d
* @return
*/
@Bean("queue_d")
public Queue queue_d(){
return QueueBuilder.nonDurable(DEAD_LETTER_QUEUE_D).build();
}
Step 4: bind the queue and switch
/**
* Delay queue (a) bind (xa) normal switch (exchange_X)
* @param queue_a
* @param exchange_X
* @return
*/
@Bean
public Binding queueABindingExchangeX(@Qualifier("queue_a") Queue queue_a,
@Qualifier("exchange_X") DirectExchange exchange_X){
return BindingBuilder.bind(queue_a).to(exchange_X).with("xa");
}
/**
* Delay queue (b) bind (xb) normal switch (exchange_X)
* @param queue_b
* @param exchange_X
* @return
*/
@Bean
public Binding queueBBindingExchangeX(@Qualifier("queue_b") Queue queue_b,
@Qualifier("exchange_X") DirectExchange exchange_X){
return BindingBuilder.bind(queue_b).to(exchange_X).with("xb");
}
/**
* Dead letter queue (d) bind (dead) dead letter switch (exchange_Y)
* routing key Consistent with the x-dead-letter-routing-key, the message queue cannot receive messages
* @param queue_d
* @param exchange_Y
* @return
*/
@Bean
public Binding queueDBindingExchangeY(@Qualifier("queue_d") Queue queue_d,
@Qualifier("exchange_Y") DirectExchange exchange_Y){
return BindingBuilder.bind(queue_d).to(exchange_Y).with("dead");
}
}
Step 5: write a controller
Don't lead the wrong package, huh
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Date;
@Slf4j
@RestController
@RequestMapping("/mq")
public class RabbitmqController {
@Autowired
private RabbitTemplate rabbitTemplate;
@GetMapping("/producer01/{message}")
public String producer01(@PathVariable String message) {
/**
* First parameter: switch name
* The second parameter: routing key
* Third parameter: message
*/
rabbitTemplate.convertAndSend("exchange_X","xa","Delay 10 s Message:"+message);
rabbitTemplate.convertAndSend("exchange_X","xb","Delay 20 s Message:"+message);
log.info("[Current time]:{},send message{}Give me two TTL queue",new Date().toString(),message);
return "News["+message+"]Send to[ queue_a]Queue success";
}
}
Step 6: write consumer (listener)
Don't lead the wrong package, huh
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.util.Date;
@Slf4j
@Component
public class DeadLetterQueueCustomer {
// monitor
@RabbitListener(queues = "queue_d")
public void receiveD(Message message, Channel channel){
byte[] body = message.getBody();
String msg = new String(body);
log.info("[Current time]:{},The message received is {}",new Date().toString(),msg);
}
}
2. Implement the delay queue in [X. 2] (set the delay sending time of each message)
==Messages in rabbitmq are queued, which will lead to A problem. If A message A with an expiration time of 20s is sent to the queue in the first second, and A message B with an expiration time of 5s is sent to the opposite column in the second second second, B will still be in the queue after 10s. Because there is an A in front of B and the expiration time of A is greater than B, B needs to wait for A to expire.
For example, to avoid the above situation, we need to install rabbitmq_delayed_message_exchange-3.9.0 j in rabbitmq to solve the above problems (the installation is as follows).
Principle: the principle is to set a delay waiting time for each message when sending a message. When the delay time is up, the message is forwarded through exchange, so that the message arriving at the queue is a message without delay. The delay effect of the message is realized by waiting before sending the message.
Download the plug-in and install it in the docker container
Download Plug-in address: https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/tag/3.9.0
Step 1: download the plug-in and transfer it to linux using sftp tool, and then copy the plug-in to the docker container (under the plugins directory of rabbitmq) through the docker cp /home/plugins/rabbitmq_delayed_message_exchange-3.9.0.ez myrabbit:/plugins command
Step 2: enter the container docker exec -it myrabbit /bin/bash, and switch to the plugins directory
Step 3: install the rabbitmq plugins enable rabbitmq_delayed_message_exchange
chmod 777 /plugins/rabbitmq_delayed_message_exchange-3.9.0.ez
Step 4: exit the container with exit, and then restart the container docker restart myrabbit
Model
When the producer sends a message, set a delayed sending time. When the time is up, the exchange_X forwards the message to the queue_c, and the consumer consumes the message accordingly.
Create switch
import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
/**
* @author HuXuehao (StudiousTiger)
* @desc The purpose of this class is to write a plug-in based delay queue
* @date 2021/10/28
*/
@Configuration
public class rabbitmqConfig {
// Declare switch name
public static final String EXCHANGE_X = "exchange_X";
// Declaration queue name
public static final String QUEUE_C = "queue_c";
/**
* Declare exchange_ 10. CustomExchange represents a custom type
* First parameter: switch name
* Second parameter: switch type
* The third parameter: false indicates no persistence
* The fourth parameter: false indicates that it will not be deleted automatically
* @return
*/
@Bean("exchange_X")
public CustomExchange exchange_X(){
Map<String, Object> arguments = new HashMap<>();
// The type of delay is direct
arguments.put("x-delayed-type","direct");
// Type is a delayed message
return new CustomExchange(EXCHANGE_X,"x-delayed-message",false,false);
}
Create queue_c
/**
* Claim delay queue_c
* @return
*/
@Bean("queue_c")
public Queue queue_c(){
return QueueBuilder.nonDurable(QUEUE_C).build();
}
Queue queue_c binding exchange_X
/**
* Delay queue (queue_c) binding (xc) normal switch (exchange_X)
* @param queue_c
* @param exchange_X
* @return
*/
@Bean
public Binding queueCBindingExchangeX(@Qualifier("queue_c") Queue queue_c,
@Qualifier("exchange_X") CustomExchange exchange_X){
return BindingBuilder.bind(queue_c).to(exchange_X).with("xc").noargs();
}
Write producer
We use the convertandsend (string exchange, string routingkey, object message, messagepostprocessor, messagepostprocessor) method to send messages. The last parameter is a functional interface, so we can use the Ramdas expression.
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessagePostProcessor;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Date;
@Slf4j
@RestController
@RequestMapping("/mq")
public class RabbitmqController {
@Autowired
private RabbitTemplate rabbitTemplate;
@GetMapping("/producer02/{message}/{timeout}")
public String producer02(@PathVariable String message,@PathVariable int timeout) {
// Ramdas expression
MessagePostProcessor messagePostProcessor = (Message msg)->{
// Get the message properties through msg, and then set the Delay time (ms)
msg.getMessageProperties().setDelay(timeout*1000);
// Message postProcessMessage(Message var1) has a return value
return msg;
};
/**
* First parameter: switch name
* The second parameter: routing key
* Third parameter: message
* The fourth message: the message post processor, which we can use to set the expiration time of the message
*/
rabbitTemplate.convertAndSend("exchange_X","xc","delay"+timeout+"s Message:"+message,messagePostProcessor);
log.info("[Current time]:{},send message{}Give me two queue_c queue",new Date().toString(),message);
return "News["+message+"],"+timeout+"s Send to after[ queue_c]Queue success";
}
}
Write consumer
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.util.Date;
@Slf4j
@Component
public class DeadLetterQueueCustomer {
// monitor
@RabbitListener(queues = "queue_c")
public void receiveD(Message message, Channel channel){
byte[] body = message.getBody();
String msg = new String(body);
log.info("[Current time]:{},The message received is {}",new Date().toString(),msg);
}
}
Send the following requests separately
localhost:8080/mq/producer02/studious_timeout_20s/20
localhost:8080/mq/producer02/studious_timeout_5s/5
Perfect solution!
✈ ❀ I hope I can give you an extraordinary experience ☂ ✿…
