1, Spring Cloud Stream

1. What is it

• Shielding the differences between the underlying message middleware, reducing the switching cost and unifying the programming model of message;

1. Officially, Spring Cloud Stream is a framework for building message driven microservices.
2. The application interacts with the binder object in Spring Cloud Stream through inputs or outputs. We configure binding, and the binder object of Spring Cloud Stream is responsible for interacting with the message middleware. Therefore, we only need to figure out how to interact with Spring Cloud Stream to facilitate the use of message driven methods.
3. By using Spring Integration to connect the message broker middleware to realize message event driven. Spring Cloud Stream provides personalized automatic configuration implementation for some vendors' message middleware products, citing three core concepts: publish / subscribe, consumption group and message partition.
4. Currently, only RabbitMQ and Kafka are supported.

Spring Cloud Stream It is a highly scalable event driven microservice framework for building connection with shared messaging system. The framework provides a flexible programming model, which is based on established and familiar Spring idioms and best practices, including three core concepts of publish / subscribe, consumer group and message partition supporting persistence.
Spring Cloud Stream Reference Documentation
Spring Cloud Stream Chinese instruction manual

2. Design idea

2.1 standard MQ

• Information content is transmitted between producers / consumers through message media
• Messages must go through a specific message channel
• How are messages in the message channel consumed and who is responsible for receiving and processing them (SubscribableChannel, a sub interface of the message channel, is subscribed by the MessageHandler message processor)

2.2 Cloud Stream

• For example, we use RabbitMQ and Kafka. Due to the different architectures of the two message middleware, RabbitMQ has exchange and Kafka has Topic and Partitions;
• The differences of these middleware lead to some problems in our actual project development. If we use one of the two message queues and the business requirements behind it, I want to migrate to another message queue. At this time, it is undoubtedly disastrous. A lot of things have to be pushed down and redone because it is coupled with our system, At this time, Spring Cloud Stream provides us with a decoupling method.
  
• Why can Stream unify the underlying differences?

1. Without the concept of binder, when our Spring Boot application needs to directly interact with message oriented middleware, there will be great differences in the implementation details due to the different original intentions of each message oriented middleware;
2. By defining the binder as the middle layer, the isolation between application and message middleware details is perfectly realized. By exposing the unified Channel channel to the application, the application does not need to consider a variety of different message middleware implementations.
3. By defining Binder as the middle layer, the isolation between application and message middleware details is realized.

• Binder

1. The further encapsulation of message oriented middleware by Stream can make the code layer insensitive to the middleware, and even dynamically switch the middleware (Rabbitmq to Kafka), which makes the development of micro services highly decoupled, and services can pay more attention to their own business processes;
2. Binder can generate binding. Binding is used to bind producers and consumers of message containers. It has two types: INPUT and OUTPUT. INPUT corresponds to consumers and OUTPUT corresponds to producers.
   

2.3 the message communication mode in stream follows the publish / subscribe mode

• Broadcast Topic

1. In RabbitMQ, it is exchange (switch)
2. In Kakfa, it is topic

3. Spring Cloud Stream standard process routine

3.1 Binder

It is convenient to connect middleware to shield differences;

3.2 Channel

Channel is an abstraction of Queue. In the message communication system, it is the medium for storage and forwarding. The Queue is configured through channel;

3.3 Source and Sink

Simply understood, the reference object is the Spring Cloud Stream itself. Publishing messages from the Stream is output, and receiving messages is input.

4. Coding API and common annotations

form	explain
Middleware	Middleware. At present, it only supports RabbitMQ and Kafaka
Binder	Binder is the encapsulation between application and message middleware. At present, RabbitMQ and Kafaka's binder are implemented. Binder can easily connect middleware and dynamically change message types (corresponding to Kafaka's Topic and RabbitMQ's Exchange). These can be realized through configuration files
@Input	The annotation identifies the input channel through which messages received enter the application
@Output	Annotations identify the output channel through which published messages leave the application
@StreamListener	Listening queue is used for receiving messages from the consumer's queue
@EnableBinding	It refers to the binding of channel and exchange

5. Message driven producers

5.1 create a new module cloud stream rabbitmq provider8801

5.2 POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud-2020</artifactId>
        <groupId>com.qs.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-stream-rabbitmq-provider8801</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!--Hot deployment-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

5.3 YML

server:
  port: 8801

spring:
  application:
    name: cloud-stream-provider
    
  cloud:
    stream:
      # Configure the service information of rabbitmq to bind here
      binders:
        # Represents the name of the definition, which is used for integration with binding
        defaultRabbit:
          # Message component type
          type: rabbit
          # Set the environment configuration related to rabbitmq
          environment:
            spring:
              rabbitmq:
                host: 192.168.137.122
                port: 5672
                username: guest
                password: guest
      # Integration of services
      bindings:
        # This name is the name of a channel
        output:
          # Represents the Exchange name definition to use
          destination: studyExchange
          # Set the message type, json this time, and "text/plain" for text
          content-type: application/json
            # Set the specific settings of the message service to be bound
            binder: defaultRabbit

eureka:
  # Configure Eureka registration on the client
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka
  instance:
    # Set the heartbeat interval (30 seconds by default)
    lease-renewal-interval-in-seconds: 2
    # If the interval of 5 seconds is exceeded now (the default is 90 seconds)
    lease-expiration-duration-in-seconds: 5
    # Displays the host name in the information list
    instance-id: send-8801.com
    # The access path becomes an IP address
    prefer-ip-address: true

5.4 main startup

@SpringBootApplication
public class StreamMQMain8801 {

    public static void main(String[] args) {
        SpringApplication.run(StreamMQMain8801.class, args);
    }
}

5.5 IMessageProvider

public interface IMessageProvider {

    String send();
}

// It can be understood as the definition of a message sending pipeline
@EnableBinding(Source.class)
public class MessageProviderImpl implements IMessageProvider {

    // Message sending pipeline
    @Resource
    private MessageChannel output;

    @Override
    public String send() {
        String serial = UUID.randomUUID().toString();
        // Create and send messages
        this.output.send(MessageBuilder.withPayload(serial).build());
        System.out.println("serial: " + serial);
        return serial;
    }
}

5.6 SendMessageController

@RestController
public class SendMessageController {

    @Resource
    private IMessageProvider messageProvider;

    @GetMapping(value = "/sendMessage")
    public String sendMessage() {
        return messageProvider.send();
    }
}

5.7 testing

http://192.168.137.155:15672/#/
http://localhost:8801/sendMessage

6. Message driven consumers

6.1 create a new module cloud stream rabbitmq consumer 8802

6.2 POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud-2020</artifactId>
        <groupId>com.qs.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-stream-rabbitmq-consumer8802</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!--Hot deployment-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

6.3 YML

server:
  port: 8802

spring:
  application:
    name: cloud-stream-consumer
    
  cloud:
    stream:
      # Configure the service information of rabbitmq to bind here
      binders:
        # Represents the name of the definition, which is used for binding integration
        defaultRabbit:
          # Message component type
          type: rabbit
          # Set the environment configuration related to rabbitmq
          environment:
            spring:
              rabbitmq:
                host: 192.168.137.122
                port: 5672
                username: guest
                password: guest
      # Integration of services
      bindings:
        # This name is the name of a channel
        input:
          # Represents the Exchange name definition to use
          destination: studyExchange
          # Set the message type. This time, it is an object json. If it is text, set "text/plain"
          content-type: application/json
          # Set the specific settings of the message service to be bound
          binder: defaultRabbit

eureka:
  client: # Configure Eureka registration on the client
    service-url:
      defaultZone: http://localhost:7001/eureka
  instance:
    # Set the heartbeat interval (30 seconds by default)
    lease-renewal-interval-in-seconds: 2
    # If the interval of 5 seconds is exceeded now (the default is 90 seconds)
    lease-expiration-duration-in-seconds: 5
    # Displays the host name in the information list
    instance-id: receive-8802.com
    # The access path becomes an IP address
    prefer-ip-address: true

6.4 main startup

@SpringBootApplication
public class StreamMQMain8802 {
    
    public static void main(String[] args) {
        SpringApplication.run(StreamMQMain8802.class, args);
    }
}

6.5 ReceiveMessageListener

@Component
@EnableBinding(Sink.class)
public class ReceiveMessageListener {

    @Value("${server.port}")
    private String serverPort;

    @StreamListener(Sink.INPUT)
    public void input(Message<String> message) {
        System.out.println(serverPort + " Received message: " + message.getPayload());
    }
}

6.6 testing

http://localhost:8801/sendMessage

7. Consumer groups and persistence

7.1 message repeat consumption

• For example, in the following scenario, when we deploy the order system in a cluster, we will get the order information from RabbitMQ. If an order is obtained by two services at the same time, it will cause data errors. We have to avoid this situation. At this time, we can use the message grouping in the Stream to solve the problem.
• Note that multiple consumers in the same group in the Stream are competitive, which can ensure that messages will only be consumed once by one of the applications. Different groups can be fully consumed (repeated consumption). There will be competition in the same group, and only one of them can be consumed.
  
• In order to achieve high availability and load balancing, distributed microservice applications actually deploy multiple instances;
• When a producer sends a message to a specific microservice, it only wants to be consumed once. According to the above example of starting two applications, although they belong to the same application, the message is consumed twice. To solve this problem, the concept of consumption group is provided in Spring Cloud Stream.
• grouping

# Different groups
# 8002
group: groupA
# 8003
group: groupB

# Same group
# 8002
group: groupQS
# 8003
group: groupQS

For multiple micro service instances in the same group, only one will get the message at a time;

7.2 message persistence

1. Stop 8802 and 8803, and remove groupQS from 8802
2. 8801 sends 4 messages
3. Restart 8802, no message received
4. Restart 8803 and receive 4 messages
5. 8801 sends a message
6. 8802 and 8803 receive messages at the same time