Using custom protocol to transfer data from Python to Netty

This article customizes a data protocol to send data to Netty receiver through Python language
The reason why two different languages are used also shows that the transmission between data is independent of language. As long as the sender and receiver abide by the same protocol
Protocols are ubiquitous, such as the HTTP protocol related to the network, the RESP protocol used to send commands to Redis, the data transmission between Dubbo consumers and providers, the message transmission between RocketMQ consumers and servers, the protocol used to obtain stack information using jstack command in the JVM, etc. there must be a set of related protocols between them, For data transmission
Everything is protocol. No matter how many protocols there are in the world, there are no more than a few common protocols. The decoder of related common protocols has been provided by default in Netty

// Based on fixed length
FixedLengthFrameDecoder
// Use the fixed length field to store the length of the content
LengthFieldBasedFrameDecoder
// Based on newline
LineBasedFrameDecoder
// Based on separator
DelimiterBasedFrameDecoder

For example, HTTP protocol uses content length in its request header to indicate the length of the request body
In fact, Dubbo,RocketMQ,Redis, etc. are also implemented with similar ideas. Such protocols are very common and practical

Therefore, Netty implements such a protocol decoder by default, that is, the LengthFieldBasedFrameDecoder decoder. It was introduced in the previous article and will not be introduced here

The customized protocol in this article is similar to it, as shown below

The whole protocol consists of two parts: request head and request body. The request head is used to store the length of the request body, and the request body is the real storage of data

The next step is to demonstrate the code

First, take a look at the Python side (as a client, it is used to send data)

#! /usr/bin/env python
# -*- coding:utf-8 -*-

from socket import *
import struct
import json

class Book(object):

    def __init__(self, addr, year):
        self.addr = addr
        self.year = year

    def addr(self):
        return self.addr

    def keys(self):
        return ['addr', 'year']

    def __getitem__(self, item):
        return getattr(self, item)


if __name__ == '__main__':
    client = socket(AF_INET, SOCK_STREAM)
    # Connect server
    client.connect(('127.0.0.1', 8082))

    book = Book('Chengdu', 2021)
    
    # Convert the object book to JSON format data
    json = json.dumps(dict(book))
    # Convert JSON format data into byte type for network transmission
    body = bytes(json, 'utf-8')
    # Calculate the length of the actual valid data (body)
    body_len = len(body)

    # Length of body stored in head
    # I represents an integer, that is, the len gt h of the body is stored in an integer size space; And > means big end storage. As for what is big end and small end storage, you can Google
    head = struct.pack('>I', body_len)
    # Send byte data to the server according to our custom protocol format
    client.sendall(head + body)
    

Take another look at the Netty receiver

import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;

public class Server {


    public static void main(String[] args) throws Exception {

        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup(8);

        ServerBootstrap serverBootstrap = new ServerBootstrap();

        try {

            serverBootstrap.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .handler(new LoggingHandler(LogLevel.INFO))
                    .childHandler(new ChannelInitializer<NioSocketChannel>() {
                        @Override
                        protected void initChannel(NioSocketChannel ch) {
                            ChannelPipeline channelPipeline = ch.pipeline();
                            // According to our custom protocol, the valid data is decoded and decoded into valid byte data
                            channelPipeline.addLast(new LengthFieldBasedFrameDecoder(1024, 0, 4, 0, 4));
                            // Convert the decoded byte data into an entity object
                            channelPipeline.addLast(new MyDecoder());
                            // Working with entity objects
                            channelPipeline.addLast(new ServerHandler());

                        }
                    });

            ChannelFuture channelFuture = serverBootstrap.bind("127.0.0.1", 8082).sync();
            channelFuture.channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
}

import com.alibaba.fastjson.JSON;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;


public class MyDecoder extends SimpleChannelInboundHandler<ByteBuf> {


    @Override
    protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception {

			// Entering this channelRead0 method indicates that the previous LengthFieldBasedFrameDecoder decoder has decoded the valid body data. This channelRead0 method obtains the valid body data
        byte[] bytes = new byte[msg.readableBytes()];
        msg.readBytes(bytes);

        String json = new String(bytes, "UTF-8");
        MyModel v = JSON.parseObject(json, MyModel.class);
        ctx.fireChannelRead(v);
    }
}

import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;


public class ServerHandler extends SimpleChannelInboundHandler<MyModel> {

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, MyModel msg) throws Exception {
        System.out.println(msg.getAddr());
    }
}

The functions of the above Netty code are shown in the figure below


First run the Netty server, then run the Python client, and you can see the data received by the Netty server on the console

Simply recorded a video

Python sends data to Netty receiver

Personal site
Language bird

official account

Tags: Python Java Netty Redis

Posted on Fri, 01 Oct 2021 18:11:21 -0400 by dyntryx