RabbitMQ explains the concept of message communication

1. RabbitMQ message communication architecture

When it comes to message communication, we may first think of email, QQ, wechat, SMS and other communication methods. These communication methods have sender, receiver and a container for storing offline messages. But these communication modes are different from the RabbitMQ communication model we are going to talk about. For example, compared with the mail communication mode, the mail server is based on the POP3/SMTP protocol, the communication parties need to specify clearly, and the content of the sent mail has a fixed structure. The RabbitMQ server is based on AMQP protocol, which does not need to specify the sender and receiver explicitly, and the message sent does not have a fixed structure. It can even store binary data directly, and it can store offline messages as well as mail server. The most important thing is that RabbitMQ can not only route one-to-one, but also one to many To broadcast.

The following figure roughly shows the process of RabbitMQ message communication:

2. Producers and consumers

There are two main roles in RabbitMQ communication: producer and consumer. Similar to the sender and receiver of mail communication.

First of all, we need to make sure that rabbitimq server can not generate data, just like its name - message middleware, which is a middleman used to deliver messages. The producer generates the creation message and then publishes it to the proxy server (RabbitMQ), while the consumer obtains the message from the proxy server (instead of directly looking for the producer for the message). In practical application, the producer and the consumer can also switch roles, so when our application connects to the RabbitMQ server, it must be clear whether I am a producer or not Consumer.

3. news

The producer creates the message and publishes it to the RabbitMQ server, so what is the message?

The message here is divided into two parts: payload and content tag.

  • Payload: can be anything, an array, a collection, or even binary data. RabbitMQ doesn't care what data you send, just send it.
  • Content tag: describes the valid content that RabbitMQ uses to determine who will get the message. In the mail communication mentioned above, the sender address and the receiver address must be specified clearly, while RabbitMQ based on AMQP protocol sends the message to interested consumers through the content tag attached to the message sent by the producer.

Later we will analyze what the tag is in detail. Here we just need to know that the producer will create the message and set the tag. Pay attention to the big picture at the top. Generally speaking, the producer will set a label when creating a message, but there will be no label when it is transmitted to the consumer. Unless you state in the effective content who is the producer, the consumer generally does not know who generates the message.

4. channel

The producer generates the message, and then publishes it to the rabbitmq server. Before publishing, the producer must first connect to the server, that is, to establish a TCP connection between the application and the rabbitmq server. Once the connection is established, the application can create an AMQP channel.

The channel is a virtual connection established in the "real" TCP connection. AMQP commands are sent out through the channel. Each channel will be assigned a unique ID (AMQP library will help you remember the ID). These actions are completed through the channel whether they are publishing messages, subscribing to queues or receiving messages.

One might ask, why not send AMQP commands directly over a TCP connection?

The reason is efficiency. For the operating system, it is very expensive to establish and destroy a TCP session every time. However, in the actual system, such as the e-commerce double 11, there are thousands of connections per second in the peak period. Generally speaking, the operating system has several restrictions on establishing a TCP connection, so it will encounter bottlenecks.

By introducing the concept of channel, we can create more than N channels on a TCP connection, which can not only send commands, but also ensure the privacy of each channel. We can think of it as a fiber-optic cable.

5. Switches and queues

Switches and queues are part of the RabbitMQ server. We know that producers will send messages to the RabbitMQ server. After entering the server, they first enter the switch part, and then the switch binds the messages to the corresponding queues according to the content labels attached to the messages. Let's first look at what a queue is:

  • In the place where the message is held, the message sent by the producer to the RabbitMQ server will wait for consumer consumption in the queue.
  • The queue is the last destination in the RabbitMQ server (unless the message enters the black hole, the concept of black hole will be introduced below).
  • Queues can be load balanced, we can add a bunch of consumers, and then let RabbitMQ distribute messages evenly in a circular way.

Now that you know what the queue is, how does the message get to the queue? Yes, it's through the switch.

When a message enters the RabbitMQ server, it will first send the message to the switch, and then the switch will bind the message to the corresponding queue according to the specific routing algorithm and the content label of the message. There are four kinds of switches in AMQP protocol: direct, fanout, topic and headers. Each switch implements different routing algorithms, which also corresponds to several different ways of RabbitMQ work. This is the key point. The blog will give a detailed introduction later.

6. Virtual host

In the big picture above, I drew virtual host A and virtual host B, indicating that there are multiple virtual hosts in RabbitMQ server, so what is virtual host?

First of all, we raise the question that a RabbitMQ must not only serve one application, so how can multiple applications use the RabbitMQ server at the same time to ensure that they do not conflict with each other?

The answer is to use virtual host. Virtual host is actually a mini RabbitMQ server. It has its own switch and queue. More importantly, virtual host has its own permission mechanism. A server can create multiple virtual hosts. When we use RabbitMQ server, we only need to correspond an application to a virtual host. This logical separation between the instances can ensure that different applications can deliver messages safely.

The default virtual host is "/".

7. Simple example

After introducing some basic concepts in RabbitMQ message communication process, let's have a real experience through a code example.

This is a Maven project. First, let's look at the pom.xml file: import the AMQP client dependency

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.ys.rabbitmq</groupId>
  <artifactId>RabbitMQTest</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>war</packaging>

  <name>RabbitMQTest Maven Webapp</name>
  <!-- FIXME change it to the project's website -->
  <url>http://www.example.com</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>com.rabbitmq</groupId>
      <artifactId>amqp-client</artifactId>
      <version>3.4.1</version>
    </dependency>

  </dependencies>

</project>

Producer:

package com.ys.simple;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.ys.utils.ConnectionUtil;

/**
 * Create by hadoop
 */
public class Send {
    private final static String QUEUE_NAME = "hello";

    public static void main(String[] args) throws Exception{
        //1. Get connection
        Connection connection = ConnectionUtil.getConnection("192.168.146.251",5672,"/","guest","guest");
        //2. Claim channel
        Channel channel = connection.createChannel();
        //3. Declaration (creation) queue
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);
        //4. Define message content
        String message = "hello rabbitmq ";
        //5. Publish message
        channel.basicPublish("",QUEUE_NAME,null,message.getBytes());
        System.out.println("[x] Sent'"+message+"'");
        //6. Closing channels and connections
        channel.close();
        connection.close();
    }
}

Consumer:

package com.ys.simple;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.QueueingConsumer;
import com.ys.utils.ConnectionUtil;


/**
 * Create by hadoop
 */
public class Recv {

    private final static String QUEUE_NAME = "hello";

    public static void main(String[] args) throws Exception{
        //1. Get connection
        Connection connection = ConnectionUtil.getConnection("192.168.146.251",5672,"/","guest","guest");
        //2. Claim channel
        Channel channel = connection.createChannel();
        //3. Declaration queue
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);
        //4. Define consumers for the queue
        QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
        //5. Listening queue
        channel.basicConsume(QUEUE_NAME,true,queueingConsumer);
        //6. Get message
        while (true){
            QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();
            String message = new String(delivery.getBody());
            System.out.println(" [x] Received '" + message + "'");
        }
    }

}

Tool class: ConnectionUtil

package com.ys.utils;

import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * Create by hadoop
 */
public class ConnectionUtil {

    public static Connection getConnection(String host,int port,String vHost,String userName,String passWord) throws Exception{
        //1. Define connection factory
        ConnectionFactory factory = new ConnectionFactory();
        //2. Set server address
        factory.setHost(host);
        //3. Set port
        factory.setPort(port);
        //4. Set virtual host, user name and password
        factory.setVirtualHost(vHost);
        factory.setUsername(userName);
        factory.setPassword(passWord);
        //5. Get connection through connection factory
        Connection connection = factory.newConnection();
        return connection;
    }
}

 

Published 8 original articles, praised 0, visited 7207
Private letter follow

Tags: RabbitMQ Maven Apache Hadoop

Posted on Wed, 12 Feb 2020 22:10:25 -0500 by james_holden