Implementation of message push system with Websocket+RabbitMQ

1, There are two modes for users to get new message notifications

  • Take the initiative to ask the system after logging in

  • When online, the system actively pushes new messages to the receiver

Imagine that the user's notification message and new notification reminder data are placed in the database, and the database reads and writes frequently. If the message volume is large and the DB pressure is high, there may be a data bottleneck. At this time, the message queue RabbitMQ can be introduced for traffic peak shaving.

Send a WebSocket message to the specified user and handle the situation that the other party is not online

  • If the receiver is online, send the message directly;

  • Otherwise, the message will be stored in redis, and the unread message will be actively pulled after the user goes online.

2, Websocket+RabbitMQ message push architecture diagram

As can be seen from the figure, the basic process of the message notification system is that client A requests the server core module. The core module produces A message to the message queue, and then the server message module consumes the message. After consumption, it pushes the message to client B. the process is very simple without many skills. The only subtlety is the processing of the message module.

  Generally, there is also a setting item for setting whether to receive messages in the "personal center" to meet the personalized needs of users.

  Our current process is somewhat ingenious. Originally, consumers should request "user message settings" before sending a message, and the user is set to receive it before generating a message. In our current process, consumers don't pay attention to user settings, put all messages in the "queue" and let the main process filter them, so that each producer doesn't have to deal with them separately, and there is also less network interaction.

3, Type of message notification

Almost every site has a message notification system, so the importance of the notification system is self-evident. The notification system seems simple, but it is actually complex. Then this article mainly explains the design and implementation of common message notification systems, including database design, logical relationship analysis, etc.

Common station notification categories:

  • Announcement
  • Remind remind

    • Resource subscription reminder "notify me when there are updates, comments and other events on the resources I follow"
    • Resource publishing reminder "notify me when my published resources have comments, collections and other events"
    • The system reminds "the platform may do something to your resources according to some algorithms and rules, and you will receive the system notification."
  • Private mail Mailbox

The above three messages have their own characteristics and different implementations. Among them, the "reminder" notification is the most complex:

Notification event:

Notification event is when a user generates a payment behavior on the website or application. If you want to give a user a notification that the system has received her payment, you need to define the "payment behavior" as a notification event and save the notification event in the "notification event table" for asynchronous processing by the notification system. The notification system will continuously process the data in the notification event table to analyze who should be notified and who should not be notified of each event.

Notification event table "notify_event"

Record the notification event information generated by each user behavior

The structure of the table is as follows:

id: {type: 'integer', primaryKey: true, autoIncrement:true} 
userID: {type: 'string', required: true} //User ID
action: {type: 'string', required: true} //Actions, such as: donation / update / comment / collection
objectID: {type: 'string', required: true}, //Object ID, such as Article ID;
objectType: {type: 'string', required: true} //Type of object, such as person, article, activity, video, etc;
createdAt: {type: 'timestamp', required: true} //Creation time;

User behavior definition

"action" refers to user behavior, such as: like, comment, like, donate and collect; Generally speaking, we define a user behavior as a notification type, so the user behavior must be defined in advance.

It is internally defined by the message system and provides an interface for the background for notification settings. As follows:

notify_action_type := ["donated","conllected","commented","updated"]

Object type definition

"objectType" refers to the type of the object under the action of user behavior. In short, it is the resource type, such as item, article, comment, commodity, video, picture and user.

It is internally defined by the message system and provides an interface for the background for notification settings. As follows:

notify_object_type := ["project","comment"]

4, Message notification system considerations

4.1. How to maintain a long connection when Nginx agent webSocket is automatically disconnected in 60s

When using nginx to proxy websocket, it is found that after the client and server shake hands successfully, if there is no data interaction within 60s, the connection will be automatically disconnected.

Proxy in location in nginx.conf file_ read_ The timeout is disconnected for 60s by default. You can set it larger. You can set it to the time you need. I set it here as ten minutes (600s)

nginx is configured as follows:

server {
        listen 80;
        #error_page 502 /static/502.html;

        location /static/ {
            root /home/chenming/Carrefour/;
            expires 7d;

        location / {
            proxy_pass_header Server;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Scheme $scheme;
            proxy_http_version  1.1;
            proxy_set_header    Upgrade    "websocket";
            proxy_set_header    Connection "Upgrade";
            proxy_read_timeout 600s; 

After setting according to the above method, we can find that if there is no data interaction within 10 minutes, the websocket connection will be automatically disconnected, so there is still a problem in this method. If my page stays for more than 10 minutes and there is no data interaction, the connection will still be disconnected, so we need to combine the websocket heartbeat mechanism at the same time

4.2 heartbeat activation mechanism of WebSocket

The heartbeat mechanism sends a data packet to the server every other period of time to tell the server that it is still alive. At the same time, the client will confirm whether the server is still alive. If it is still alive, it will send a data packet back to the client to confirm that the server is still alive. Otherwise, the network may be disconnected. Reconnection is required.

WebSocket long connection requires a stable reconnection mechanism in the case of weak network environment and temporary disconnection of the network, so as to ensure that the client and server can reconnect and continue communication when the network is unstable.

On the basis of nginx extending the timeout time, the front end sends heartbeat packets within the timeout time and refreshes the reread time. See the following code for the specific implementation of the front end (the code here includes the implementation process of the whole websocket of the front end, in which the content of sending heartbeat packets is highlighted in red):

// websocket connection
var websocket_connected_count = 0;
var onclose_connected_count = 0;
function newWebSocket(){
    var websocket = null;
    // Judge whether the current environment supports websocket
            var ws_url ="wss://"+domain+"/updatewebsocket";
            websocket = new WebSocket(ws_url);
        Tip("not support websocket");
    // Callback method for successful connection establishment
    websocket.onopen = function(e){
        heartCheck.reset().start();   // After the connection is successfully established, reset the heartbeat detection
        Tip("connected successfully")
    // An error occurred in the connection. When the connection error occurs, the connection will continue to be attempted (5 attempts)
    websocket.onerror = function() {
        console.log("onerror Connection error")
        if(websocket_connected_count <= 5){
    // Callback method to receive message
    websocket.onmessage = function(e){
        console.log("Received the message")
        heartCheck.reset().start();    // If the message is obtained, it indicates that the connection is normal and the heartbeat detection is reset
        var message =;
           //Perform the operation of receiving the message, generally refreshing the UI
    // Accept the callback method when the server closes the connection
    websocket.onclose = function(){
        Tip("onclose Disconnect");
    // Listen for window events. When the window is closed, actively disconnect the websocket connection to prevent closing the window before the connection is disconnected, and the server side reports an error
    window.onbeforeunload = function(){
    // Heartbeat detection detects the connection status at regular intervals. If it is connected, it will actively send a message to the server to reset the maximum connection time between the server and the client. If it has been disconnected, it will initiate reconnection.
    var heartCheck = {
        timeout: 55000,        // Send a heartbeat every 9 minutes, which is slightly smaller than the connection time set on the server. When it is close to disconnection, reset the connection time by means of communication.
        serverTimeoutObj: null,
        reset: function(){
            return this;
        start: function(){
            var self = this;
            this.serverTimeoutObj = setInterval(function(){
                if(websocket.readyState == 1){
                    console.log("Connection status, send message to keep connected");
                    heartCheck.reset().start();    // If the message is obtained, it indicates that the connection is normal and the heartbeat detection is reset
                    console.log("Disconnected state, trying to reconnect");
            }, this.timeout)

4.3. The RabbitMQ consumer must catch exceptions

The Bug I encountered was that there was no Emoj expression filtering on the front-end input, resulting in JPA insertion error, and the consumer program did not try catch exceptions, which eventually led to the abnormal termination of the consumer program.

5, User defined HandshakeInterceptor, which is used to prohibit unlisted users from connecting to WebSocket

package cn.zifangsky.stompwebsocket.interceptor.websocket;
import cn.zifangsky.stompwebsocket.common.Constants;
import cn.zifangsky.stompwebsocket.common.SpringContextUtils;
import cn.zifangsky.stompwebsocket.model.User;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.server.ServerHttpRequest;
import org.springframework.http.server.ServerHttpResponse;
import org.springframework.stereotype.Component;
import org.springframework.web.socket.WebSocketHandler;
import org.springframework.web.socket.server.HandshakeInterceptor;
import javax.servlet.http.HttpSession;
import java.text.MessageFormat;
import java.util.Map;
 * Customize {@ link org.springframework.web.socket.server.HandshakeInterceptor} to realize "login is required to connect to WebSocket"
public class AuthHandshakeInterceptor implements HandshakeInterceptor {
    private final Logger logger = LoggerFactory.getLogger(getClass());
    public boolean beforeHandshake(ServerHttpRequest serverHttpRequest, ServerHttpResponse serverHttpResponse, WebSocketHandler webSocketHandler, Map<String, Object> map) throws Exception {
        HttpSession session = SpringContextUtils.getSession();
        User loginUser = (User) session.getAttribute(Constants.SESSION_USER);
        if(loginUser != null){
            logger.debug(MessageFormat.format("user{0}Request establishment WebSocket connect", loginUser.getUsername()));
            return true;
            logger.error("You are not logged in to the system. Connection is prohibited WebSocket");
            return false;
    public void afterHandshake(ServerHttpRequest serverHttpRequest, ServerHttpResponse serverHttpResponse, WebSocketHandler webSocketHandler, Exception e) {

Reference link:

Practical scheme of Web side message notification mechanism

Design of message notification system model

When Nginx proxy webSocket is disconnected automatically in 60s, how to maintain a long connection

Possible bug s after the consumer channel is actively disconnected

Tags: RabbitMQ websocket

Posted on Fri, 17 Sep 2021 19:35:03 -0400 by gazolinia