Introduction to RabbitMQ installation and operation principle

preface

This article will discuss how to install rabbitmq on linux, basic commands, common configurations, and how to use rabbitmq on java clients; We will compare some differences between activeMQ and the implementation of common usage principles.

Introduction to RabbitMQ

RabbitMQ Is an open source AMQP Implementation, server-side Erlang Language, support a variety of clients. It is used to store and forward messages in distributed systems, and performs well in ease of use, scalability, high availability and so on.
Lightweight and easy to deploy on site and in the cloud. It supports multiple messaging protocols. MQ can be deployed in distributed and federated configurations to meet large-scale and high availability requirements.
Realized The version of AMQP is 0.9.0, and activeMQ implements version 1.0
Basic documents:
Messaging that just works — RabbitMQ Official website: Messaging that just works — RabbitMQ
RabbitMQ Tutorials — RabbitMQ RabbitMQ course: RabbitMQ Tutorials — RabbitMQ
Source code git address: github.com
This can be clearly felt on the official website. Compared with activeMQ, it is highly available   More complete support on Distributed

characteristic

Includes asynchronous messaging, developer experience, distributed deployment, enterprise + cloud readiness, tools and plug-ins, management and monitoring

 

Including the documents of the client and server,

  install

Install to Centos 7 or other linux   Official documents have corresponding versions   We can choose

Downloading and Installing RabbitMQ — RabbitMQ

 

  matters needing attention

  • The version of erlang must match the RabbitMQ version
  • Install as root (or sudo)
  • You must be able to access the Internet
Installation steps
  • Install the required dependent library EPEL
yum install epel-release
  • Install Erlang Solutions repository
wget https://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm rpm -Uvh erlang-solutions-1.0-1.noarch.rpm
  • Install erlang
yum -y install erlang
erlang The installation process will take some time. Please wait patiently.
  • Install dependent package socat
yum -y install socat
  • Installing RabbitMQ in RPM mode
rpm -Uvh https://dl.bintray.com/rabbitmq/all/rabbitmq-server/3.7.9/rabbitmq- server-3.7.9-1.el7.noarch.rpm
  • Running rabbitmq server service
Enable service
systemctl enable rabbitmq-server
Turn on the service (if it is occupied by other programs) amqp Default port 5672 , stop the program (if possible) or modify it rabbitmq
server Port)
systemctl start rabbitmq-server
Shut down service
systemctl stop rabbitmq-server
  • Location description of RabbitMQ program, configuration file, log and data directory after installation
rabbitmq These directory files will be given in the startup log:
less /var/log/rabbitmq/rabbit@localhost.log

/etc/rabbitmq There is no configuration file by default.
Official website description connection: File and Directory Locations — RabbitMQ

Basic configuration

RabbitMQ has a set of default configurations that can meet daily development needs. If it needs to be modified, you need to create a configuration file yourself

In other words, rabbitmq does not require us to modify the configuration.

Some commonly used commands

  • Viewing rabbitmq status     systemctl status rabbitmq

  •   Where is rabbitmq view the directory of rabbitmq

Including some places to store logs

 

  •   Through less rabbit@localhost.log You can query the log directory and so on

To modify the configuration, create a configuration file: /etc/rabbitmq/rabbitmq.conf
Configuration file example:
https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.conf.example
Configuration Item Description:
https://www.rabbitmq.com/confifigure.html#confifig-items

Rabbit port

rabbitMQ will bind some ports. After installation, you need to add these ports to the firewall.

15674 port: the stomp client port based on websocket is opened when the plug-in web stomp is enabled

15675 port: mqtt client port based on websocket. When web mqtt is opened

You can see the following command in bin

 rabbitmqctl(8) — RabbitMQ

All commands are included in the document

RabbitMQ admin console

There is a management plug-in in rabbitmq, which needs to be activated manually

rabbitmq-plugins enable rabbitmq_management
RabbitMQ There is a default user "guest" However, this user can only be accessed through this machine by default. To allow other machines to access, you need to create a new user and assign permissions to it.
#Add user 
rabbitmqctl add_user admin admin 
#Assign administrative rights to users 
rabbitmqctl set_user_tags admin administrator 
#Assign resource permissions to users 
https://www.rabbitmq.com/rabbitmqctl.8.html#set_permissions rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"

User role classification of RabbitMQ

none
Cannot access management plugin
management
Users can AMQP Anything done, plus:
  • List the virtual hosts that you can log in through AMQP
  • View the queues, exchanges and bindings in your virtual hosts
  • View and close your channels and connections
  • View "global" statistics about your own virtual hosts, including the activities of other users in these virtual hosts.

policymaker

management Anything you can do plus:
  • View, create, and delete policies and parameters to which your virtual hosts belong
monitoring
management Anything you can do plus:
  • List all virtual hosts, including those that they cannot log in to
  • View connections and channels for other users
  • View node level data, such as clustering and memory usage
  • View real global statistics about all virtual hosts

 

administrator
policymaker and monitoring Anything you can do plus:
  • Create and delete virtual hosts
  • View, create, and delete users
  • View create and delete permissions
  • Close connections for other users

Accessing the web administration console

http://ip:15672

Log in, including port, etc., and connection node name. Open channels and   Various conditions of the channel

Virtual host

Client use

RabbitMQ Tutorials — RabbitMQ

Provide examples for use and operation

  Client dependency

<dependency>
     <groupId>com.rabbitmq</groupId> 
    <artifactId>amqp-client</artifactId>
     <version>5.7.3</version> 
</dependency>

Usage scenarios interact with queues

Interact with queues

Here, a producer and consumer can directly use ConnectionFactory and create a Channel channel

  • Create connection factory
  • Set connection properties
  • Get connection from connection factory
  • Create channels from links
  • Declare (create) a queue. RabbitMQ will be created only if the queue does not exist. It is not allowed to declare two queues with the same queue name and different properties, otherwise an error will be reported
  • send message
public class Producer {

	public static void main(String[] args) throws Exception {
		// 1. Create connection factory
		ConnectionFactory factory = new ConnectionFactory();
		// 2. Set connection properties
		factory.setHost("localhost");
		factory.setPort(5672);
		factory.setUsername("admin");
		factory.setPassword("admin");
		// factory.setVirtualHost(virtualHost);

		try (
				// 3. Get connection from connection factory
				Connection connection = factory.newConnection("producer");
				// 4. Create channels from links
				Channel channel = connection.createChannel();) {

			/**
			 * 5,Declare (create) a queue. RabbitMQ will be created only if the queue does not exist. It is not allowed to declare two queues with the same queue name and different properties, otherwise an error will be reported
			 *
			 * queueDeclare Parameter Description:
			 * 
			 * @param queue
			 *            Queue name
			 * @param durable
			 *            Is the queue persistent
			 * @param exclusive
			 *            Whether it is exclusive, that is, whether it is private. If it is true, the current queue will be locked and other channels cannot be accessed. It will be automatically deleted when the connection is closed, which is not controlled by the properties of persistence and automatic deletion
			 * @param autoDelete
			 *            Whether to delete automatically. Whether to delete automatically after the last consumer disconnects
			 * @param arguments
			 *            Queue parameters, setting the validity period of the queue, the maximum length of messages, the life cycle of all messages in the queue, and so on
			 */
			channel.queueDeclare("queue1", false, false, false, null);

			// Message content
			String message = "Hello World!";
			// 6. Send message
			channel.basicPublish("", "queue1", null, message.getBytes());
			System.out.println("Send message:" + message);

		}
	}
}

On the consumer side

  • Create connection factory
  • Set connection properties
  • Get connection from connection factory
  • Create channels from links
  • Declare (create) a queue. RabbitMQ will be created only if the queue does not exist. It is not allowed to declare two queues with the same queue name and different properties, otherwise an error will be reported
  • Define the callback after receiving the message
  • Open queue consumption  
public class Consumer {

	public static void main(String[] args) throws Exception {
		// 1. Create connection factory
		ConnectionFactory factory = new ConnectionFactory();
		// 2. Set connection properties
		factory.setHost("rabbitmq.study.com");
		factory.setUsername("admin");
		factory.setPassword("admin");

		String queueName = "queue1";

		try (
				// 3. Get connection from connection factory
				Connection connection = factory.newConnection("consumer");
				// 4. Create channels from links
				Channel channel = connection.createChannel();) {
			/**
			 * 5,Declare (create) a queue. RabbitMQ will be created only if the queue does not exist. It is not allowed to declare two queues with the same queue name and different properties, otherwise an error will be reported
			 *
			 * queueDeclare Parameter Description:
			 * 
			 * @param queue
			 *            Queue name
			 * @param durable
			 *            Is the queue persistent
			 * @param exclusive
			 *            Whether it is exclusive, that is, whether it is private. If it is true, the current queue will be locked and other channels cannot access it,
			 *            It will be automatically deleted when the connection is closed, which is not controlled by the properties of persistence and automatic deletion. It is generally used when a queue is bound to a switch
			 * @param autoDelete
			 *            Whether to delete automatically. Whether to delete automatically after the last consumer disconnects
			 * @param arguments
			 *            Queue parameters, setting the validity period of the queue, the maximum length of messages, the life cycle of all messages in the queue, and so on
			 */
			channel.queueDeclare(queueName, false, false, false, null);

			// 6. Define the callback after receiving the message
			DeliverCallback callback = new DeliverCallback() {
				public void handle(String consumerTag, Delivery message) throws IOException {
					System.out.println("Message received:" + new String(message.getBody(), "UTF-8"));
				}
			};

			// 7. Open queue consumption
			channel.basicConsume(queueName, true, callback, new CancelCallback() {
				public void handle(String consumerTag) throws IOException {
				}
			});

			System.out.println("Start receiving messages");
			System.in.read();

		}
	}
}

  After running, you can see the non persistent messages on the web management console

The data is still different from activemq. It is persistent by default. When there are no consumers, the messages in this queue will always exist.

Work queue

Consumers have multiple situations   Work Queue when the message processing in the queue is relatively time-consuming, we can open multiple messengers to consume and process messages in parallel.

 

RabbitMQ Messages will be sent to consumers one by one in a polling manner. Sometimes, some consumers are short of resources, and it processes messages slower than other consumers. And RabbitMQ Without knowing the situation of consumers, it still polls and sends messages to consumers. At this time, it may cause message backlog on slow consumers, messages can not be processed in time, and fast consumers are idle. To avoid this, we can prefetch count Reduce( =1 Too conservative, too low prefetch will affect throughput performance).

As long as there is no confirmation, the message always exists.

  Different from the above, the message sender is used in the code here

public static void main(String[] args) throws Exception {
		// 1. Create connection factory
		ConnectionFactory factory = new ConnectionFactory();
		// 2. Set connection properties
		factory.setHost("localhost");
		factory.setUsername("admin");
		factory.setPassword("admin");

		String queueName = "queue1";

		try (
				// 3. Get connection from connection factory
				Connection connection = factory.newConnection("Consumer 1");
				// 4. Create channels from links
				Channel channel = connection.createChannel();

				// You can create multiple channels for the same connection, or create channels for different connections to form multiple consumers
				// Connection connection2 = factory.newConnection("consumer 2");
				// Channel channel2 = connection2.createChannel();
				Channel channel2 = connection.createChannel();) {

			// 5. Declare (create) a queue. RabbitMQ will be created only if the queue does not exist. It is not allowed to declare two queues with the same queue name and different properties, otherwise an error will be reported
			channel.queueDeclare(queueName, false, false, false, null);

			// When it takes more time for consumers to process a message, reduce pre sending to prevent the message from being processed in time
			channel.basicQos(1); // accept only one unack-ed message at a time

			// 6. Define the callback after receiving the message
			DeliverCallback callback = (consumerTag, message) -> {
				System.out.println(consumerTag + " Message received:" + new String(message.getBody(), "UTF-8"));
			};

			// 7. Open queue consumption
			channel.basicConsume(queueName, true, callback, consumerTag -> {
			});

			// Second consumer
			channel2.basicQos(1); // Prefetch only one message
			channel2.basicConsume(queueName, true, callback, consumerTag -> {
			});

			System.out.println("Start receiving messages");


		}
	}

You can create several channels to fetch messages. The default is 256 prefetched messages. The value here is arbitrarily set to 1. Fetch once for each channel

Using in spring

Just add the following configuration in application.yml

spring:
  rabbitmq:
    host: rabbitmq.study.com
    port: 5672
    username: admin
    password: admin  
   # listener:
   #   simple:
   #      prefetch: 1

Add corresponding when using   RabbitListener   consumer

	@RabbitListener(queues = "hello")
	public void receive(String in) {
		System.out.println(" [x] Received '" + in + "'");
	}

producer

34. Messaging (spring.io)   

  • Here, configure a bean of the Queue we want to operate on. The spring rabbit MQ framework will get these beans from the container at startup,
  • And create these queue s, exchange and binding to the rabbitmq server.
  • This is done in the RabbitAdmin.initialize() method.
  • The completed work is: channel.queueDeclare("hello",false, false, false, null);
  • We can also manually create the queue, exchange and binding we need through the AmqpAdmin.declareXXX(xxx) method.
public class HelloWorldProducer {
	
	//Instructions for amqp in spring boot: 
	//https://docs.spring.io/spring-boot/docs/2.1.6.RELEASE/reference/html/boot-features-messaging.html#boot-features-amqp

	/*
	 * [Note: here, configure a bean of the Queue that we want to operate on. The spring rabbit MQ framework will get these beans from the container at startup,
	 * And create these queue s, exchange and binding to the rabbitmq server.
	 * This is done in the RabbitAdmin.initialize() method.
	 * The completed work is: channel.queueDeclare("hello",false, false, false, null);
	 * We can also manually create the queue, exchange and binding we need through the AmqpAdmin.declareXXX(xxx) method.
	 * 
	 * @Autowired private AmqpAdmin amqpAdmin;
	 * 
	 * public void send() { 
	 * 		... 
	 * 		this.amqpAdmin.declareQueue(new Queue("hello"));
	 * 		... 
	 * }
	 */
	@Bean
	public Queue hello() {
		return new Queue("hello");
	}

	// @Autowired  
	// private AmqpAdmin amqpAdmin; / / used for queue, exchange and binding management

	@Autowired
	private RabbitTemplate template;

	@Autowired
	private Queue queue;

	@Scheduled(fixedDelay = 1000)   //Send messages multiple times at regular intervals
	public void send() {
		String message = "Hello World!";
		this.template.convertAndSend(queue.getName(), message);
		System.out.println(" [x] Sent '" + message + "'");
	}

	public static void main(String[] args) throws Exception {
		SpringApplication.run(HelloWorldProducer.class, args);
		System.in.read();
	}
}

It also simplifies writing code for us

Then set the prefetch in spring or add it in the configuration

	@Bean
	public SimpleRabbitListenerContainerFactory myFactory(SimpleRabbitListenerContainerFactoryConfigurer configurer,
			ConnectionFactory connectionFactory) {
		SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
		configurer.configure(factory, connectionFactory);
		// factory.setMessageConverter(myMessageConverter());
		factory.setPrefetchCount(1);
		return factory;
	}

publish/subscribe

Exchange 

RabbitMQ The core idea of the messaging model is that the producer will never send any message directly to the queue. In fact, usually the producer does not even know whether the message will be delivered to any queue.
On the contrary, the producer can only send messages to the exchange. The exchange is a very simple thing. While receiving messages from the producer, the exchange pushes the messages to the queue. The exchange must know exactly how to process the messages it receives. Should messages be appended to a specific queue or to many queues? ? or should be discarded. These rules exchange Type definition.
In the previous queue, use an empty string as exchange
Exchange types
have : direct, topic, headers and fanout
  • fanout: fan switch
It routes all messages sent to the switch to all queues bound to the switch
  • Direct: direct switch
It routes messages to those BindingKey and RoutingKey In an exact match queue
  • Topic: topic switch
And direct Similar, but it can be fuzzy matched by wildcards
  • headers: header switch
It does not rely on the matching rules of routing keys to route messages, but according to the information in the sent message content headers Property.
headers This type of exchanger has poor performance and is not practical.
Nameless exchanger : "" , in the previous example, we did not create a switch, but used the system default nameless switch.
channel.basicPublish("", "hello", null, message.getBytes());
establish exchange
channel.exchangeDeclare(String exchange, String type)
channel.exchangeDeclare(EXCHANGE_NAME, "fanout");

send message

channel.basicPublish(EXCHANGE_NAME, "", null, message.getBytes("UTF-8"));

When exchange is specified, routingkey is not specified

On the consumer side

Create a temporary queue, the name is automatically generated (unique), the connection is disconnected and automatically deleted;

Bind the queue to exchange. The routingKey specified during binding is also called bindingkey. routingKey is useless in fanout exchange. This is mainly used to bind the queue and establish an association relationship with exchange  

public static void main(String[] argv) throws Exception {
		ConnectionFactory factory = new ConnectionFactory();
		factory.setHost("localhost");
		factory.setPort(5672);
		factory.setUsername("admin");
		factory.setPassword("admin");

		try (Connection connection = factory.newConnection();
				Channel channel = connection.createChannel();
				Connection connection2 = factory.newConnection();
				Channel channel2 = connection2.createChannel();) {

			channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
			// Create a temporary queue. The name is automatically generated (unique), disconnected and deleted
			String queueName = channel.queueDeclare().getQueue();
			// Bind the queue to exchange. The routingKey specified during binding is also called
			// Bindingkey. routingKey is useless in the fanout switch.
			channel.queueBind(queueName, EXCHANGE_NAME, "");

			DeliverCallback deliverCallback = (consumerTag, message) -> {
				System.out.println(consumerTag + " Message received:" + new String(message.getBody(), "UTF-8"));
			};
			channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
			});

			// Second consumer
			String queueName2 = channel2.queueDeclare().getQueue();
			channel2.queueBind(queueName2, EXCHANGE_NAME, "");
			channel2.basicConsume(queueName2, true, deliverCallback, consumerTag -> {
			});

			System.out.println("Start receiving messages");
			System.in.read();
		}
	}

In spring

@Configuration
public class PubSubConfiguration {

	@Bean
	public FanoutExchange fanout() {
		return new FanoutExchange("spring-logs");
	}

	@Configuration
	public static class ReceiverConfig {

		@Bean
		public Queue autoDeleteQueue1() {
			return new AnonymousQueue();
		}

		@Bean
		public Queue autoDeleteQueue2() {
			return new AnonymousQueue();
		}

		@Bean
		public Binding binding1(FanoutExchange fanout, Queue autoDeleteQueue1) {
			return BindingBuilder.bind(autoDeleteQueue1).to(fanout);
		}

		@Bean
		public Binding binding2(FanoutExchange fanout, Queue autoDeleteQueue2) {
			return BindingBuilder.bind(autoDeleteQueue2).to(fanout);
		}
	}
}

Then everything else has to be the same.

routing

In the consumer, separate different messages and send them to different queues corresponding to the consumer using the routing key

 

 

  The difference among consumers is the use of different routingkey s   orange   To distinguish black green is equivalent to classification

channel.exchangeDeclare(EXCHANGE_NAME, "direct");
			String queueName = channel.queueDeclare().getQueue();
			channel.queueBind(queueName, EXCHANGE_NAME, "orange");

			DeliverCallback deliverCallback = (consumerTag, message) -> {
				System.out.println(consumerTag + " Message received:" + new String(message.getBody(), "UTF-8"));
			};
			channel.basicConsume(queueName, true, deliverCallback, consumerTag -> {
			});

			// Second consumer
			String queueName2 = channel2.queueDeclare().getQueue();
			channel2.queueBind(queueName2, EXCHANGE_NAME, "black");
			channel2.queueBind(queueName2, EXCHANGE_NAME, "green");
			channel2.basicConsume(queueName2, true, deliverCallback, consumerTag -> {
			});

On the provider side, you only need to add  

channel.basicPublish(EXCHANGE_NAME, routingKey, null, message.getBytes("UTF-8"));

In spring, this is very similar to the previous exchange

	@Bean
		public Binding binding1a(DirectExchange direct, Queue autoDeleteQueue1) {
			return BindingBuilder.bind(autoDeleteQueue1).to(direct).with("orange");
		}

		@Bean
		public Binding binding1b(DirectExchange direct, Queue autoDeleteQueue1) {
			return BindingBuilder.bind(autoDeleteQueue1).to(direct).with("black");
		}

Add to configuration when Binding

Topic

 

  Using topic mode exchange   Unlike before, fuzzy matching is adopted here to expand the granularity and multiple dimensions

Binding binding

We have created a fan out switch and a queue. Now we need to tell you exchange Send a message to the queue. The relationship between exchange and queue is called binding.
channel.queueBind(queueName, "logs", "");
channel.queueBind(String queue, String exchange, String routingKey)

  RPC mode

  The way of remote procedure call: the caller on the client and server, and the program on the remote server return data

mq itself is a data transfer station, which can be realized by mq.

A queue needs to be generated accordingly

public static void main(String[] argv) {

		try (RPCClient fibonacciRpc = new RPCClient()) {

			for (int i = 0; i < 32; i++) {

				String i_str = Integer.toString(i);

				System.out.println(" [x] Requesting fib(" + i_str + ")");

				String response = fibonacciRpc.call(i_str);

				System.out.println(" [.] Got '" + response + "'");

			}

		} catch (IOException | TimeoutException | InterruptedException e) {
			e.printStackTrace();
		}
	}

	public String call(String message) throws IOException, InterruptedException {

		final String corrId = UUID.randomUUID().toString();

		String replyQueueName = channel.queueDeclare().getQueue();

		AMQP.BasicProperties props = new AMQP.BasicProperties.Builder().correlationId(corrId).replyTo(replyQueueName)
				.build();

		channel.basicPublish("", requestQueueName, props, message.getBytes("UTF-8"));

		final BlockingQueue<String> response = new ArrayBlockingQueue<>(1);

		String ctag = channel.basicConsume(replyQueueName, true, (consumerTag, delivery) -> {

			if (delivery.getProperties().getCorrelationId().equals(corrId)) {
				response.offer(new String(delivery.getBody(), "UTF-8"));
			}

		}, consumerTag -> {});

		String result = response.take();
		channel.basicCancel(ctag);
		return result;
	}

	public void close() throws IOException {
		connection.close();
	}

Call through the queue to generate data and the message publisher.

Tags: RabbitMQ Distribution

Posted on Wed, 27 Oct 2021 00:00:31 -0400 by Xproterg^vi