netty network programming 2

Netty network programming

1. IO concept

IO refers to input stream and output stream to support the external transmission and exchange of data by JAVA applications. The transfer in destination can be file, network, memory, etc.

With the increasing demand for transmission efficiency, java has gradually evolved three generations of IO models, namely BIO, NIO and AIO.

BIO synchronous Blocking I/O

Before Java 1.4, this kind of transmission can only be realized through inputStream and OutputStream. This is a blocking IO model. It's good to deal with file systems with few connections. If you deal with thousands of network services, this blocking model will cause a large number of threads to be occupied and the server can't bear higher concurrency.

NIO synchronous Non Blocking I/O

In order to solve this problem, NIO was introduced after Java 1.4. It communicates through two-way pipes and supports non blocking, which solves the problem of thread occupation caused by network transmission. Netty uses this communication model at the bottom.

AIO Asynchronous Blocking I/O

The non blocking implementation of NIO relies on the selector to cycle the pipeline state. If there are many pipelines at the same time, the performance will be affected. Therefore, JAVA 1.7 introduces asynchronous non blocking IO to replace the selector through asynchronous callback. This change is obvious in windows, but not in linux. Now most JAVA systems are deployed through linux, so AIO is not widely used. So we next focus on the comparison between BIO and NIO.

2. Differences between bio and NIO models

The biggest difference between the two groups of models is blocking and non blocking, and what is the so-called blocking? How to solve non blocking?

Blocking model BIO

In the blocking model, after the client establishes a connection with the server, three events will occur in sequence

1. The client writes data into the stream. (blocked)

2. Transmit to the server through network channel (TPC/IP). (blocked)

3. The server reads.

During this process, the thread on the server side will block and wait until the data is sent, and then execute step 3.

If the client does not write data in step 1 or the network transmission delay in step 2 is too high, the server thread blocking time will be longer. Therefore, more concurrency means that more threads are needed to support it.

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-h8vizxzl-1637830678928) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027171435717. PNG)]

In the BIO model, 1-to-1 threads are used to wait for the completion of steps 1 and 2. In NIO, a Selector is assigned to check whether the conditions for executing step 3 are met. If they are met, the thread will be notified to execute step 3 and process the business. In this way, the delay of steps 1 and 2 is independent of the business thread.

Non blocking model NIO

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-kfzfv5bq-1637830678929) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027172411009. PNG)]

3.NIO basic components Channel and Buffer

In BIO API, InputStream and outPutStream are used for input and output. In NIO, a two-way communication pipeline is used to replace them. A channel must rely on a Buffer for communication.

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-lbchckvn-1637830678931) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211027173241991. PNG)]

The pipeline has more features than the stream, such as non blocking, out of heap memory mapping, zero copy and so on.

Note: different pipes support different functions, and not all pipes support the above characteristics

Buffer buffer

All pipelines rely on Buffer buffer, which must be understood first.

Buffer definition

The so-called buffer is a data container, which maintains an array for storage. Buffer buffer does not support storing any data. It can only store some basic types, and even strings cannot be stored directly.

[the external chain image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-4inahlhw-1637830678932) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211027191315538. PNG)]

In the ordinary development process, the byte type is used more

Buffer internal structure

An array is maintained inside the Buffer, and there are three properties we need to pay attention to, namely:

Capacity: capacity, that is, the size of the internal array. Once declared, this value cannot be changed

**Position: position, * * current read / write position. The default is 0. 1 will be added for each bit read or written

Limit: limit, that is, the maximum value that can be read and written. It must be less than or equal to capacity

With capacity as the capacity limit, why should there be a limit? The reason is that when writing data to the Buffer, it may not be full, and the limit is used to mark which position is written to, and it will not exceed the standard when reading.

If the reading exceeds the standard, a BufferUnderflowException will be reported

Similarly, bufferoverflow exception will also be reported if the write exceeds the standard

[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-agudl7yv-1637830678933) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-202110281013306633. PNG)]

Buffer core usage

  • allocate: declare a buffer of the specified size. Position is 0 and limit is the capacity value

  • Wrap: wrap a Buffer based on the array. position is 0 and limit is the capacity value

flip operation

To prepare for reading, set position to 0 and limit to the original position value

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-phdh2oxv-1637830678934) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027191111301. PNG)]

clear operation

To prepare for writing, set position to 0 and limit to the capacity value

Note: clear does not actually clear the data

rewind operation

To prepare for writing, set position to 0 and limit unchanged. It can also be used to read. However, since the limit is not changed, unwritten empty data may be read, resulting in errors in the read data.

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ncmxgfjl-1637830678934) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027191208788. PNG)]

mark operation

Add a tag so that the subsequent call to reset will return the position to the tag. Common scenarios, such as replacing a section of content

[the external link image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-dsipt2oy-1637830678935) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211027191234771. PNG)]

At present, we know that there are four values: mark, position, limit and capacity, which are equal to the following rules:

0 < = mark < = position < = limit < = capacity

Channel pipeline

Defined by the java.nio.channels package, the Channel represents the open connection between the IO source and the target. The Channel is similar to the traditional "flow", but the Channel itself cannot directly access data, and the Channel can only interact with the Buffer. The Channel is mainly used to transmit data from one side of the Buffer to entities on the other side (such as files, sockets...), and vice versa; The Channel is the conduit for accessing IO services. Through the Channel, we can access the I/O services of the operating system with minimal overhead; By the way, the Buffer is the endpoint that sends and receives data within the Channel

Channel definition

channel pipes are used to connect files, network sockets, etc. It can perform read and write I/O operations at the same time. It is called a two-way pipe. It has two states: connected and closed. It is open when creating the pipe. Once closed, it will report a ClosedChannelException when calling the I/O operation. Whether the pipeline is open can be determined by the ispen method of the pipeline.

Common channel pipes

The commonly used channels in general development include FileChannel, datagram channel, ServerSocketChannel and SocketChannel

FileChannel file pipeline

Fixed name suggests that it is used to operate files. In addition to normal operations, it also supports the following features:

  • Supports reading and writing to the specified area of the file

  • Out of heap memory mapping. When reading and writing large files, it can be directly mapped out of the JVM declaration memory to improve the reading and writing efficiency.

  • Zero copy technology directly transfers data to a channel through transferFrom or transferTo, greatly improving performance.

  • Lock the specified area of the file to prevent other programmers from accessing it

At present, FileChannel can only be opened indirectly through the stream, such as inputStream.getChannel() and outputStream.getChannel(). The pipeline opened through the input stream can only fetch, while the pipeline opened by outputStream can only write. Otherwise, NonWritableChannelException and NonReadableChannelException will be thrown respectively.

If you want the pipeline to support both read and write, you must use RandomAccessFile read and write mode.

FileChannel example:

FileChannel channel = new RandomAccessFile(file_name,"rw").getChannel();

ByteBuffer buffer=ByteBuffer.allocate(1024); 

int count = channel.read(buffer);

The read method writes data to the Buffer until the Buffer is full or the data has been read. count returns the number of reads, - 1 indicates that the read has been completed.

Technologies such as out of heap memory mapping and zero copy will be described in detail in the NIO based zero copy in subsequent chapters

Datagram channel UDP socket pipe

udp is a connectionless protocol. Datagram channel provides services for this protocol to receive messages from clients.

udp implementation steps are as follows:

// 1. Open the pipe
DatagramChannel channel = DatagramChannel.open();
// 2. Binding port
channel.bind(new InetSocketAddress(8080));
ByteBuffer buffer = ByteBuffer.allocate(8192);
// 3. Receive the message. If the client has no message, it will block and wait
channel.receive(buffer); 

nc -vu 127.0.0.1 8080 this command can send messages to udp

TCP socket pipe

TCP is a protocol with connection. It can communicate only after a connection is established. This requires the following two pipes:

  • **ServerSocketChannel: * * used to establish a connection with the client

  • **SocketChannel: * * used to read and write messages to clients

The implementation steps of TCP pipeline are as follows:

// 1. Open the TCP service pipeline
ServerSocketChannel channel = ServerSocketChannel.open();
// 2. Binding port
channel.bind(new InetSocketAddress(8080));
// 3. Accept the connection request sent by the client (if not, it will be blocked)
SocketChannel socketChannel = channel.accept();
ByteBuffer buffer=ByteBuffer.allocate(1024);
// 4. Read the message from the client (if not, it will be blocked)
socketChannel.read(buffer);
// 5. Write back message
socketChannel.write(ByteBuffer.wrap("Return message".getBytes()));
// 6. Close the pipe
socketChannel.close();

The TCP service telnet 127.0.0.1 8080 can be tested through the command

The above example receives a message, returns it to the client, and then closes. It can only handle the request of one client, and then the whole service ends. If we want to process multiple requests, we can add a loop to receive requests, and then allocate a sub thread to process each client request.

ServerSocketChannel channel = ServerSocketChannel.open();
channel.bind(new InetSocketAddress(8080));
while(true){
  SocketChannel socketChannel = channel.accept();
  // Use child threads to process requests
  new Thread(()->handle(socketChannel)).start();
}

Processing client requests

private void handle(SocketChannel socketChannel){
    ByteBuffer buffer=ByteBuffer.allocate(1024);
    // 1. Read the message from the client (if not, it will be blocked)  
     socketChannel.read(buffer);
     // Return message
     socketChannel.write(ByteBuffer.wrap("Return message".getBytes()));
       // Close the pipe
     socketChannel.close();
 }

So far, we have completed a simple but complete request response model. In this model, each client connection will have a child thread to process. This thread will block in the read method until the message is read. Therefore, it is not difficult to infer that this is a typical BIO blocking model.

So how to implement the non blocking NIO model in the pipeline? Let's continue.

4.NIO multiplexer

Selector working model

Above, we implemented a blocking model with a pipeline. In that model, after the server establishes a connection, it will immediately allocate a thread to wait for the message to arrive. Because I don't know when the message will arrive at the client, it is mainly blocked and waiting.

Can I wait for the message to arrive and process it in the allocation thread? This requires the Selector to appear. Just set the pipe to non blocking mode and register with the Selector. You will be notified when the message arrives.

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-mm5xyaz3-1637830678936) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211027180215621. PNG)]

In the figure, the pipeline on the left is not registered with the Selector and needs to block waiting messages. The pipeline on the right has been registered with the Selector and will allocate threads when messages arrive. Thread blocking is avoided.

How does the selector implement this function? Before solving this problem, let's take a look at the core components in the selector

Selector core components

There are three selector core components: selectablechannel, selector and selector key.

Not all channels support registering with selectors. Only SelectableChannel subclasses can register with selectors. When the pipeline is registered with the Selector, a key will be returned, and the associated pipeline can be obtained through this key. Next, we will introduce the functions of the three components:

[external link image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-2ph5lkv3-1637830678937) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027181609893. PNG)]

SelectableChannel pipeline

There are two core functions. configureBlocking sets the blocking mode type. The default value is true, which represents the blocking mode. Therefore, it must be set to false before registering with the selector. The second is to call the register method to register with the specified pipeline and specify the events to listen. Optional events are:

Connect, accept, read, write. However, not all pipelines support these four events. You can view which events are supported by the current pipeline through validOps().

Selector selector

After the pipeline is registered with the Selector, a key (Selector key) will be generated, which is maintained in the keys of the Selector. And refresh by calling the select method. If the returned number is greater than 0, it indicates that the state of a specified number of keys has changed.

  • select(): if there is a key update, return immediately. Otherwise, it will block until there is a key update.

  • **select(long): * * there is a key update. Return immediately. Otherwise, it will block when the parameter specifies milliseconds.

  • **selectNow(): * * returns immediately whether there is a key update or not

If there are key updates, you can then call selectedKeys to get the updated keyset.

SelectionKey selection key

Key is used to associate the pipeline and selector, and listen to and maintain one or more events of the pipeline. The listening events can be specified during registration or changed later by calling SelectionKey.interestOps.

// Listen for read events
key=socketChannel.register(selector, SelectionKey.OP_READ);
// Listen for read and write events at the same time
key.interestOps(SelectionKey.OP_READ|OP_WRITE);

Four constants in the SelectionKey represent four events

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-gwi5mizm-1637830678937) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211027181955684. PNG)]

However, not all pipelines support these four events, such as:

ServerSocketChannel only supports OP_ACCEPT event,

SocketChannel supports OP_CONNECT,OP_READ,OP_WRITE three events

Check which events are supported by the pipeline and use the value returned by validOps(), and then perform '&' operation to judge, such as

//Indicates that the pipeline supports OP_CONNECT event listening
socketChannel.validOps()&SelectionKey.OP_CONNECT != 0

In addition, Key has the following main functions:

  1. channel() get pipeline

  2. Is the isAcceptable() pipeline in the Accept state

  3. Is the isConnectable pipe in connection ready state

  4. Is the isReadable pipeline in the read ready state

  5. Is the isWritable pipeline in write on state

  6. isValid() determines whether the key is valid. Closing the pipeline, closing the selector, and calling the cancel() method will make the key invalid.

  7. **cancel() * * cancel pipeline registration (the pipeline will not be closed directly)

Use of selectors

After knowing the three components of the Selector, I will gradually understand the Selector through a simple Demo.

Demo demo

The following is an example of UDP. Open the pipeline through datagram channel and register it with Selector. Then, the Selector listens for the status of the message.

@Test
public void selectorUdpTest() throws IOException {
   Selector selector = Selector.open();
    DatagramChannel channel = DatagramChannel.open();
    channel.bind(new InetSocketAddress(8080));
    // Set non blocking mode
    channel.configureBlocking(false);
    // 1. Registration read continuation event
   channel.register(selector, SelectionKey.OP_READ);
    while (true) {
        //2. Refresh keyset
        int count = selector.select();
        if (count > 0) {
            Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
            // 3. The sequel
           while (iterator.hasNext()) {
               SelectionKey key = iterator.next();
                //4. Process the continuation key
               handle(key);
                //5. Remove from sequel 
               iterator.remove();
            }
        }
    }
}

Process the key, get the pipe from the key and read the message

public void handle(SelectionKey key) throws IOException {
   ByteBuffer buffer = ByteBuffer.allocate(8192);
   DatagramChannel channel = (DatagramChannel) key.channel();
   channel.receive(buffer);// Read message and write to buffer
}

Use process

Description of key processes of the above examples:

  1. Register the pipeline through channel.register().
  2. Refresh the status of registered keys through selector.select().
  3. selector.selectedKeys() gets the sequel and traverses it.
  4. Process key, that is, get the pipe and read the message.
  5. Remove from selection set.

Focus on steps 4 and 5. If step 4 (i.e. reading messages in the pipeline) is not executed, the pipeline is still in the read continue state. When the selector select() method is called, it will return immediately, causing an endless loop. If step 4 is performed but step 5 is not performed, it will cause it to remain in the selection set and repeat the processing.

Keyset description

A total of three key sets are maintained in the selector. The underlying implementation is Set, so it will not be repeated:

  • All keys: all keys registered with the selector are placed here

  • Selected keys: store the keys to be continued

  • Canceledkeys: store cancelled keys

Either refreshing or closing the selector causes the keyset to change. The following figure details the key

[external link image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-hrfh7phf-1637830678938) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211027184042427. PNG)]

  • Calling select() refreshes the key, adds the ready set to the selection set, clears the cancel key set, and removes the cancelled key
  • Remove the selection set. The selection set will not be selected by the selector. You need to call Set.remove() to remove it
  • Cancel() or close the selector and close the pipe will add the key to the cancel set, but it will not be cleared immediately and will be cleared only at the next refresh.

NIO implements simple httpServer

package com.example.nettydemo.nio.http;
import java.io.*;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.Collectors;

public class SimpleHttpServer {
    private final Selector selector;
    int port;
    private Set<SocketChannel> allConnections = new HashSet<>();
    volatile boolean run = false;
    HttpServlet servlet;
    ExecutorService executor = Executors.newFixedThreadPool(5);

    public SimpleHttpServer(int port, HttpServlet servlet) throws IOException {
        this.port = port;
        this.servlet = servlet;
        ServerSocketChannel listenerChannel = ServerSocketChannel.open();
        selector = Selector.open();
        listenerChannel.bind(new InetSocketAddress(port));
        listenerChannel.configureBlocking(false);
        listenerChannel.register(selector, SelectionKey.OP_ACCEPT);
    }

    public Thread start() {
        run = true;
        Thread thread = new Thread(() -> {
            try {
                while (run) {
                    dispatch();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }, "selector-io");
        thread.start();
        return thread;
    }

    public void stop(int delay) {
        run = false;
    }

    private void dispatch() throws IOException {
        int select = selector.select(2000);
        Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
        while (iterator.hasNext()) {
            SelectionKey key = iterator.next();
            iterator.remove();
            if (key.isAcceptable()) {
                ServerSocketChannel channel = (ServerSocketChannel) key.channel();
                SocketChannel socketChannel = channel.accept();
                socketChannel.configureBlocking(false);
                socketChannel.register(selector, SelectionKey.OP_READ);
            } else if (key.isReadable()) {
                final SocketChannel channel = (SocketChannel) key.channel();
                ByteBuffer buffer = ByteBuffer.allocate(1024);
                final ByteArrayOutputStream out = new ByteArrayOutputStream();
                while (channel.read(buffer) > 0) {
                    buffer.flip();
                    out.write(buffer.array(), 0, buffer.limit());
                    buffer.clear();
                }
                if (out.size() <= 0) {
                    channel.close();
                    continue;
                }
                System.out.println("Current channel:" + channel);
                // Decode and encapsulate request
                Request request = decode(out.toByteArray());
                // Build a response and attach it to the current key
                Response response = new Response();
                key.attach(response);
                // Submit task to thread pool
                executor.submit(() -> {
                    servlet.doService(request, response);
                    // If other threads are blocked because they call the selector.select() or selector.select(long) methods,
                    // After the selector.wakeup() is called, the result will be returned immediately, and the returned value= 0
                    // If the current Selector is not blocked on the select method,
                    // Then this wakeup call will take effect the next time the select is blocked.
                    selector.wakeup();
                    key.interestOps(SelectionKey.OP_WRITE);
                });

            } else if (key.isWritable()) {
                final SocketChannel channel = (SocketChannel) key.channel();
                channel.write(ByteBuffer.wrap(encode((Response) key.attachment())));
                key.interestOps(SelectionKey.OP_READ);
                key.attach(null);
            }
        }


    }


    // Decoding Http services
    private Request decode(byte[] bytes) throws IOException {
        Request request = new Request();
        BufferedReader reader = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(bytes)));
        String firstLine = reader.readLine();
        System.out.println(firstLine);
        String[] split = firstLine.trim().split(" ");
        request.method = split[0];
        request.url = split[1];
        request.version = split[2];

        //Read request header
        Map<String, String> heads = new HashMap<>();
        while (true) {
            String line = reader.readLine();
            if (line.trim().equals("")) {
                break;
            }
            String[] split1 = line.split(":");
            heads.put(split1[0], split1[1]);
        }
        request.heads = heads;
        request.params = getUrlParams(request.url);
        //Read request body
        request.body = reader.readLine();
        return request;
    }

    //Encoding Http services
    private byte[] encode(Response response) {
        StringBuilder builder = new StringBuilder(512);
        builder.append("HTTP/1.1 ")
                .append(response.code).append(Code.msg(response.code)).append("\r\n");

        if (response.body != null && response.body.length() != 0) {
            builder.append("Content-Length: ")
                    .append(response.body.length()).append("\r\n")
                    .append("Content-Type: text/html\r\n");
        }
        if (response.headers != null && !response.headers.isEmpty()) {
            String headStr = response.headers.entrySet().stream().map(e -> e.getKey() + ":" + e.getValue())
                    .collect(Collectors.joining("\r\n"));
            builder.append(headStr + "\r\n");
        }


//      builder.append ("Connection: close\r\n");//  Close the link after execution
        builder.append("\r\n").append(response.body);
        return builder.toString().getBytes();
    }


    public abstract static class HttpServlet {
        void doService(Request request, Response response) {
            if (request.method.equalsIgnoreCase("GET")) {
                doGet(request, response);
            } else {
                doPost(request, response);
            }
        }

        abstract void doGet(Request request, Response response);

        abstract void doPost(Request request, Response response);
    }

    public static class Request {
        Map<String, String> heads;
        String url;
        String method;
        String version;
        String body;    //Request content
        Map<String, String> params;
    }

    public static class Response {
        Map<String, String> headers;
        int code;
        String body; //Return results
    }


    private static Map getUrlParams(String url) {
        Map<String, String> map = new HashMap<>();
        url = url.replace("?", ";");
        if (!url.contains(";")) {
            return map;
        }
        if (url.split(";").length > 0) {
            String[] arr = url.split(";")[1].split("&");
            for (String s : arr) {
                if (s.contains("=")) {
                    String key = s.split("=")[0];
                    String value = s.split("=")[1];
                    map.put(key, value);
                } else {
                    map.put(s, null);
                }
            }
            return map;

        } else {
            return map;
        }
    }
}

package com.example.nettydemo.nio.http;
import java.io.IOException;
import java.util.HashMap;


public class HttpServerTest {
    //Keep alive problem does not work
    public void simpleHttpTest() throws IOException, InterruptedException {
        SimpleHttpServer simpleHttpServer = new SimpleHttpServer(8080, new SimpleHttpServer.HttpServlet() {
            @Override
            void doGet(SimpleHttpServer.Request request, SimpleHttpServer.Response response) {
                System.out.println(request.url);
                response.body="hello word";
                response.code=200;
                response.headers=new HashMap<>();
                if (request.params.containsKey("short")) {
                    response.headers.put("Connection", "close");
                }else if(request.params.containsKey("long")){
                    response.headers.put("Connection", "keep-alive");
                    response.headers.put("Keep-Alive", "timeout=30,max=300");
                }
            }

            @Override
            void doPost(SimpleHttpServer.Request request, SimpleHttpServer.Response response) {

            }
        });
        simpleHttpServer.start().join();
    }

    public static void main(String[] args) throws IOException, InterruptedException {
        HttpServerTest httpServerTest = new HttpServerTest();
        httpServerTest.simpleHttpTest();
    }
}

5. NIO based zero copy

What is mmap?

The usual question is: why is RocketMQ fast? Why is Kafka fast? What is mmap?

Zero copy is one of these problems. Although there are other reasons, the topic of this paper is zero copy.

Traditional IO

Before we start talking about zero copy, we should first have a concept of the traditional IO mode.

Based on the traditional IO mode, the bottom layer is actually implemented by calling read() and write().

Read the data from the hard disk to the kernel buffer through read(), and then copy it to the user buffer; Then write to the socket buffer through write(), and finally write to the network card device.

[the external chain image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-dwjmvljh-1637830678938) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20210415102451017. PNG)]

In the whole process, there are 4 context switches and 4 copies in user state and kernel state. The specific process is as follows:

  1. The user process initiates a call to the operating system through the read() method, and the context changes from user state to kernel state
  2. The DMA controller copies the data from the hard disk to the read buffer
  3. The CPU copies the data from the read buffer to the application buffer, the context changes from kernel state to user state, and read() returns
  4. The user process initiates the call through the write() method, and the context changes from user state to kernel state
  5. The CPU copies the data in the application buffer to the socket buffer
  6. The DMA controller then copies the data from the socket buffer to the network card, switches the context from the kernel state to the user state, and writes () returns

[external link picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-9euwcrva-1637830678939) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20210415102502378. PNG)]

So, what are the user state and kernel state here? What is context switching?

In short, user space refers to the running space of user processes, and kernel space is the running space of the kernel.

If a process runs in kernel space, it is in kernel state, and if it runs in user space, it is in user state.

For the sake of security, they are isolated from each other, and the context switching between user state and kernel state is also time-consuming.

We can see from the above that a simple IO process produces four context switches, which will undoubtedly have a great impact on performance in high concurrency scenarios.

So what is DMA copy?

Because an IO operation is completed by the CPU issuing corresponding instructions, but compared with the CPU, the speed of IO is too slow, and the CPU has a lot of time waiting for Io.

Therefore, DMA (Direct Memory Access) Direct Memory Access technology is produced. In essence, it is an independent chip on the motherboard, through which the data of memory and IO devices are transmitted, so as to reduce the waiting time of CPU.

However, no matter who copies it, frequent copying takes time, which also has an impact on performance.

Traditional IO code demonstration

    @Test
    public void BufferTest() throws IOException {
        String file_name = "D:/500M.txt";
        String copy_name = "D:/500M_copy.txt";
        File outputFile = new File(copy_name);
        outputFile.delete();
        outputFile.createNewFile();
        long begin = System.nanoTime();

        try (InputStream inputStream = new FileInputStream(file_name);
             FileOutputStream outputStream = new FileOutputStream(outputFile);
             BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream);
             BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream)) {
            int c;
            while ((c = bufferedInputStream.read()) != -1) {
                bufferedOutputStream.write(c);
            }
            System.out.println((System.nanoTime() - begin) / 1.0e6);
        } catch (Exception e) {
            e.printStackTrace();
        }

    }

Zero-copy

Zero copy technology means that when a computer performs operations, the CPU does not need to copy data from one memory to another. This technology is usually used to save CPU cycle and memory bandwidth when transmitting files over the network.

So for zero copy, it is not really a process without data copy at all, but to reduce the number of user state and kernel state switches and the number of CPU copies.

Here, just talk about several common zero copy technologies.

mmap+write memory mapping

mmap+write simply means that mmap is used to replace the read operation in read+write, reducing one CPU copy.

The main implementation of mmap is to map the address of the read buffer to the address of the user buffer, and share the kernel buffer with the application buffer, so as to reduce one CPU copy from the read buffer to the user buffer.

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-nlmr28kb-1637830678940) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20210415102513344. PNG)]

In the whole process, there are 4 context switches and 3 copies in user state and kernel state. The specific process is as follows:

  1. The user process calls the operating system through the mmap() method, and the context changes from user state to kernel state
  2. The DMA controller copies the data from the hard disk to the read buffer
  3. The context changes from kernel state to user state, and the mmap call returns
  4. The user process initiates the call through the write() method, and the context changes from user state to kernel state
  5. The CPU copies the data in the read buffer to the socket buffer
  6. The DMA controller copies the data from the socket buffer to the network card, switches the context from the kernel state to the user state, and writes () returns

mmap saves a CPU copy. At the same time, because the memory in the user process is virtual and only mapped to the read buffer of the kernel, it can save half of the memory space and is more suitable for the transmission of large files.

mmap+write code demonstration

    // Memory mapping
    @Test
    public void mmapTest() throws IOException {
        String file_name = "D:/500M.txt";
        String copy_name = "D:/500M_copy.txt";
        File file = new File(copy_name);
        file.delete();
        file.createNewFile();
        long begin = System.nanoTime();
        FileChannel channel = new RandomAccessFile(file_name, "rw").getChannel();
        FileChannel copyChannel = new RandomAccessFile(copy_name, "rw").getChannel();

        //1. Create a mapping
        MappedByteBuffer mapped = channel
                .map(FileChannel.MapMode.READ_WRITE, 0, channel.size());
        copyChannel.write(mapped);//2. file kernel buffer 1 -- "copy cpu to file kernel buffer 2"
        System.out.println((System.nanoTime() - begin) / 1.0e6);
        copyChannel.close();
        channel.close();
    }

sendfile

Compared with mmap, sendfile also reduces one CPU copy and two context switches.

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-bta0tdli-1637830678941) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20210415102523461. PNG)]

Sendfile is a system call function introduced after the Linux 2.1 kernel version. Sendfile data can be directly transmitted in the kernel space by using sendfile, so the copy of user space and kernel space is avoided. At the same time, sendfile is used instead of read+write, which saves one system call, that is, two context switches.

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-vpege41c-1637830678942) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20210415102536532. PNG)]

In the whole process, there are 2 context switches and 3 copies in user state and kernel state. The specific process is as follows:

  1. The user process initiates a call to the operating system through the sendfile() method, and the context changes from user state to kernel state
  2. The DMA controller copies the data from the hard disk to the read buffer
  3. The CPU copies the data in the read buffer to the socket buffer
  4. The DMA controller copies the data from the socket buffer to the network card, switches the context from the kernel state to the user state, and the sendfile call returns

sendfile method IO data is completely invisible to user space, so it can only be applied to situations that do not require user space processing at all, such as static file servers.

sendfile+DMA Scatter/Gather

After the Linux 2.4 kernel version, sendfile is further optimized by introducing new hardware support, which is called DMA Scatter/Gather decentralized / collection function.

It records the data description information - memory address and offset in the read buffer to the socket buffer, and DMA copies the data from the read buffer to the network card according to these. Compared with the previous version, it reduces the process of one CPU copy

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-devtl0ky-1637830678942) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20210415102551221. PNG)]

In the whole process, there are two context switches and two copies in user state and kernel state. More importantly, there is no CPU copy at all. The specific process is as follows:

  1. The user process initiates a call to the operating system through the sendfile() method, and the context changes from user state to kernel state
  2. DMA controller uses scatter to copy data from hard disk to read buffer for discrete storage
  3. The CPU sends the file descriptor and data length in the read buffer to the socket buffer
  4. The DMA controller copies the data from the kernel buffer to the network card using scatter/gather according to the file descriptor and data length
  5. The sendfile() call returns, and the context switches from kernel state to user state

Like sendfile, DMA gather data is invisible to user space and requires hardware support. At the same time, the input file descriptor can only be a file, but there is no CPU copy process in the process, which greatly improves the performance.

sendfile code demonstration

    // sendFile direct transfer
    @Test
    public void transferFromTest() throws IOException {
        String file_name = "D:/500M.txt";
        String copy_name = "D:/500M_copy.txt";
        File file = new File(copy_name);
        file.delete();
        file.createNewFile();
        FileChannel channel = new RandomAccessFile(file_name, "rw").getChannel();
        FileChannel copyChannel = new RandomAccessFile(copy_name, "rw").getChannel();
        long begin = System.nanoTime();
        // sendFile
        // 2 switches 2 copies
        copyChannel.transferFrom(channel, 0, channel.size());//Copy from batch target to current pipeline
        System.out.println((System.nanoTime() - begin) / 1.0e6);
        channel.close();
        copyChannel.close();
    }

Application scenario

The two scenarios mentioned earlier: RocketMQ and Kafka both use zero copy technology.

For MQ, it is nothing more than that the producer sends data to MQ and then persists it to disk, and then the consumer reads data from MQ.

For RocketMQ, the two steps use mmap+write, while Kafka uses mmap+write to persist data and sendfile to send data.

Zero copy summary

Due to the difference between CPU and IO speed, DMA technology is produced to reduce the waiting time of CPU through DMA handling.

The traditional IOread+write mode will produce 2 DMA copies + 2 CPU copies and 4 context switches at the same time.

The mmap+write mode generates 2 DMA copies + 1 CPU copy and 4 context switches. Through memory mapping, one CPU copy is reduced, which can reduce memory use and is suitable for large file transmission.

sendfile mode is a new system call function, which generates 2 DMA copies + 1 CPU copy, but only 2 context switches. Because there is only one call, the context switching is reduced, but the user space is invisible to IO data, which is suitable for static file servers.

The sendfile+DMA gather mode generates 2 DMA copies, no CPU copies, and only 2 context switches. Although the performance is greatly improved, it needs to rely on new hardware device support.

6. Using Netty to implement simple UDP/TCP services

Next, we use Netty to implement a simple http service and udp service. Here is just a simple use. Later, we will introduce in detail how the Netty framework carries out network development

TCP case

Previously, we implemented a simple http service with native NIO. The following code is a simple http service implemented through netty

package com.example.nettydemo.netty.http;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.*;

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.charset.Charset;

public class BootstrapTest {

    // Write an Http service
    // http-->TCP
    public void open(int port) {
        ServerBootstrap bootstrap = new ServerBootstrap();
        EventLoopGroup boss = new NioEventLoopGroup(1);
        EventLoopGroup work = new NioEventLoopGroup(8);

        bootstrap.group(boss, work)
                .channel(NioServerSocketChannel.class)// Specify the pipeline to be opened for automatic registration = = "nioserversocketchannel - >
                .childHandler(new ChannelInitializer<Channel>() {//Specify sub pipe
                    @Override
                    protected void initChannel(Channel ch) {
                        ch.pipeline().addLast("decode", new HttpRequestDecoder()); // input
                        ch.pipeline().addLast("aggregator",new HttpObjectAggregator(65536));
                        ch.pipeline().addLast("encode", new HttpResponseEncoder());// Output stream
                        ch.pipeline().addLast("servlet", new MyServlet());

                    }
                });
        // NioServerSocketChannel.bind==>EventLoop.runTasks ==>ServerSocketChannel.bind()
        ChannelFuture future = bootstrap.bind(port);
        future.addListener(future1 -> System.out.println("login was successful"));

    }
    // Request header
    //  453 8192 8192 .... 2532

    private class MyServlet extends SimpleChannelInboundHandler {
        @Override
        public void channelActive(ChannelHandlerContext ctx) throws Exception {
            super.channelActive(ctx);
        }

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {
            // Request (request header)
            // Body (request body)
            if (msg instanceof FullHttpRequest) {
                FullHttpRequest request= (FullHttpRequest) msg;
                System.out.println("url:"+request.uri());
                System.out.println(request.content().toString(Charset.defaultCharset()));

                FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_0, HttpResponseStatus.OK);
                response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=utf-8");
                response.content().writeBytes("hello".getBytes());
                ChannelFuture future = ctx.writeAndFlush(response);
                future.addListener(ChannelFutureListener.CLOSE);
            }
           if (msg instanceof HttpRequest) {
                HttpRequest request = (HttpRequest) msg;
                System.out.println("Current request:" + request.uri());
            }
            if (msg instanceof HttpContent) {
                // Write file stream
                ByteBuf content = ((HttpContent) msg).content();
                OutputStream out = new FileOutputStream("D:/work/nettydemo/target/test.txt", true);
                content.readBytes(out, content.readableBytes());
                out.close();
            }
            if (msg instanceof LastHttpContent) {
                FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_0, HttpResponseStatus.OK);
                response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=utf-8");
                response.content().writeBytes("Upload complete".getBytes());
                ChannelFuture future = ctx.writeAndFlush(response);
                future.addListener(ChannelFutureListener.CLOSE);
            }
        }
    }

    public static void main(String[] args) throws IOException {
        new BootstrapTest().open(8080);
        System.in.read();
    }
}

By comparing the original NIO cases, we can see that netty can simplify the difficulty of network programming. In fact, the version of netty is more stable. After analyzing the implementation of netty, we will have a clearer understanding of netty.

UDP case

The following case is a request for proverb query from a client to the server, and the server returns a proverb case randomly. The code is as follows:

udp server

package com.example.nettydemo.netty.udp;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.DatagramPacket;
import io.netty.channel.socket.nio.NioDatagramChannel;
import io.netty.util.CharsetUtil;

import java.util.concurrent.ThreadLocalRandom;

/**
 * @Author: dinghao
 * @Date: 2021/10/27 20:48
 */
public class ChineseProverbServer {

    public void run(int port) throws InterruptedException {
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioDatagramChannel.class)
                    .option(ChannelOption.SO_BROADCAST, true)
                    .handler(new ChineseProverbServerHandler());
            b.bind(port).sync().channel().closeFuture().await();
        }finally {
            group.shutdownGracefully();
        }
    }

    public static void main(String[] args) throws InterruptedException {
        int port = 8685;
        if(args.length>0){
            try{
                port = Integer.parseInt(args[0]);
            }catch (NumberFormatException e){
                e.printStackTrace();
            }
        }
        new ChineseProverbServer().run(port);
    }

    private static class ChineseProverbServerHandler extends SimpleChannelInboundHandler<DatagramPacket> {
        // Proverb list
        private static final String[] DICTIONARY = {"As long as the Kung Fu is deep, the iron pestle is ground into a needle.","In the old days, the swallow in front of Wang Xie hall flew into the homes of ordinary people.",
                "Luoyang's relatives and friends are like asking each other. An ice heart is in the jade pot","An inch of time is an inch of gold but you can't buy that inch of time with an inch of gold",
                "The bright moon in front of the bed is suspected to be frost on the ground. Raising my head, I see the moon so bright; withdrawing my eyes, my nostalgia comes around."};

        private String nextQuote(){
            int quoteId = ThreadLocalRandom.current().nextInt(DICTIONARY.length);
            return DICTIONARY[quoteId];
        }

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, DatagramPacket msg) throws Exception {
            String req = msg.content().toString(CharsetUtil.UTF_8);
            System.out.println(req);
            if("Proverb dictionary query?".equals(req)){
                ctx.writeAndFlush(new DatagramPacket(Unpooled.copiedBuffer("Query results:" + nextQuote(), CharsetUtil.UTF_8),
                        msg.sender()));
            }
        }

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            ctx.close();
            cause.printStackTrace();
        }

    }
}


udp client

package com.example.nettydemo.netty.udp;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.DatagramPacket;
import io.netty.channel.socket.nio.NioDatagramChannel;
import io.netty.util.CharsetUtil;

import java.net.InetSocketAddress;

/**
 * @Author: dinghao
 * @Date: 2021/10/27 21:09
 */
public class ChineseProverbClient {

    public void run(int port) throws InterruptedException {
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioDatagramChannel.class)
                    .option(ChannelOption.SO_BROADCAST, true)
                    .handler(new ChineseProverbClientHandler());

            Channel channel = b.bind(0).sync().channel();
            // Broadcast UDP messages to all machines in the network segment
            channel.writeAndFlush(new DatagramPacket(Unpooled.copiedBuffer("Proverb dictionary query?", CharsetUtil.UTF_8),
                    new InetSocketAddress("255.255.255.255", port))).sync();
            if(!channel.closeFuture().await(15000)){
                System.out.println("Query timeout!");
            }
        }finally {
            group.shutdownGracefully();
        }
    }

    public static void main(String[] args) throws InterruptedException {
        int port = 8685;
        if(args.length>0){
            try{
                port = Integer.parseInt(args[0]);
            }catch (NumberFormatException e){
                e.printStackTrace();
            }
        }
        new ChineseProverbClient().run(port);
    }

    private static class ChineseProverbClientHandler extends SimpleChannelInboundHandler<DatagramPacket> {

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, DatagramPacket msg) throws Exception {
            String resp = msg.content().toString(CharsetUtil.UTF_8);
            if(resp.startsWith("Query results:")){
                System.out.println(resp);
                ctx.close();
            }
        }

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            ctx.close();
            cause.printStackTrace();
        }

    }
}


7.Netty thread model

Reactor mode

Reactor means reactor. Reactor model means that one or more inputs are passed to the processor at the same time. The processor synchronously assigns them to the processing thread corresponding to the request. The reactor mode is also called Dispatcher mode. Netty adopts the master-slave reactor model as a whole

Reactor role

There are three roles in the Reactor model:

  • Acceptor: handles new client connections and dispatches requests to the processor chain

  • Reactor: it is responsible for listening and allocating events, and assigning I/O events to the corresponding Handler.

  • Handler: event handling, such as encoding, decoding, etc

Reactor thread model

Reactor has three thread models: single reactor single thread model, single reactor multi thread model and master-slave reactor multi thread model. Netty is the last one.

Single Reactor single thread model

In this model, all request establishment, IO read-write and business processing are completed in one thread. If time-consuming operations occur in business processing, all requests will be delayed. Because they are processed synchronously by a thread.

[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-zy2av3un-1637830678943) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211111163437452. PNG)]

Single Reactor multithreading model

In order to prevent blocking caused by business processing, a thread pool will be used to process business asynchronously in the multithreading model. When the processing is completed, it will be written back to the client.

[the external chain image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (IMG vimizvqh-1637830678944) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211111164122171. PNG)]

Master slave Reactor multithreading model

Single React can never give full play to the parallel processing ability of multi-core CPU s in modern servers, so there can be multiple reactors, and there are one master and many slaves. A primary Reactor only handles connections, while multiple child reactors are used to handle IO reads and writes. Then it is handed over to the thread pool to process the business. Tomcat is implemented in this mode.

[external link image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-xjlnrvhd-1637830678944) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211111165215410. PNG)]

Netty threading model

Netty adopts the master-slave Reactor model. When the server starts, two nioeventloopgroups are created. The main Reactor corresponds to the Boss group thread group and the sub Reactor corresponds to the Worker Group thread group. One is used to receive the Tcp connection of the client and register the established connection with the Worker Group. The other is used to process the I/O related read and write operations. When the IO event is triggered, the corresponding Pipeline will process and execute the system Task and scheduled Task.

[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-6r5gqw4i-1637830678945) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211111171552172. PNG)]

The responsibilities of the thread pool used by Netty to receive client requests are as follows.

(1) receive the Tcp connection of the client and initialize the Channel parameters;

(2) notify ChannelPiepeline of the link state change event.

The responsibilities of the Reactor thread pool that Netty handles I/O operations are as follows

(1) asynchronously read the data of the opposite end of the communication and send a read event to the ChannelPipeline

(2) asynchronously send the message to the communication opposite end and call the message sending interface of ChannelPipeline;

(3) execute system call Task

(4) execute scheduled tasks, such as link idle status monitoring scheduled tasks.

By adjusting the number of threads in the thread pool and whether to share the thread pool, Netty's Reactor thread model can switch between single thread, multi thread and master-slave multi thread. This flexible configuration method can meet the personalized customization of different users to the greatest extent.

In order to improve the performance as much as possible, Neety has implemented lock free design in many places, such as serial operation within I/O threads to avoid performance degradation caused by multi-threaded competition. On the surface, the serialization design seems to have low CPU utilization and insufficient concurrency. However, by adjusting the parameters of NIO thread pool, multiple serialized threads can be started to run in parallel at the same time. This local lockless serial thread design has better performance than a queue multiple worker threads model.

NioEventLoop

Event circulator, which serves as the core of Reactor. Each NioEventLoop contains a Selector for processing IO events and two taskqueues, one for storing tasks submitted by users and one for processing scheduled tasks. There is a unique thread pool in EventLoop. It is not started by default. When a Task is triggered, it will be started and polled all the time.

The following example declares a thread group of size 1. After submit ting the task, the EventLoop will start

NioEventLoopGroup group= new NioEventLoopGroup(1);

// Submit task

group.submit(() -> System.out.println("submit:"+Thread.currentThread().getId()));

group.shutdownGracefully();// Close EventLoop gracefully

In addition to submitting tasks, the more important thing is to handle Channel related IO events. Such as pipeline registration. Calling the register method will eventually call the register method in NIO for registration, but Netty has implemented encapsulation and handled the problem of synchronization lock.

EventLoopGroup.register(Channel)

In order to safely call IO operations, Netty encapsulates all direct IO operations into a task, which is executed by the IO thread. Therefore, if we call IO through Netty, it will not return immediately

NioChannel and Channelhandler

netty encapsulates the original NIO Channel into NioChannel. Of course, the bottom layer is still calling NIO Channel. Originally, the processing of read-write events in Channel was encapsulated into Channelhandler for processing, and the concept of Pipeline was introduced. Let's first understand their usage

Note: the concepts of Channel and Pipeline still have many knowledge systems, which will be discussed later. This will give us an impression first.

Basic usage of Netty

NioChannel usage

Learn the usage of NioChannel through an example. In this example, the client connection will be received and the message will be printed out.

The complete implementation divides it into the following steps:

1. Initialize the pipeline

The initialization operation is similar to the native NIO, which is to open the pipeline, register the selector, and finally bind the port. But there is one thing to explain

All operations in NioChannel are completed in EventLoop, so you must register before binding ports.

NioEventLoopGroup boss= new NioEventLoopGroup(1);

NioServerSocketChannel channel= new NioServerSocketChannel();

boss.register(channel);

channel.bind(new InetSocketAddress(8080));// Submit task to EventLoop

2. Initialize Pipeline

Native NIO directly traverses the selection set and then processes read-write events. It is neither safe nor recommended to directly process read-write events in Netty. Instead, ChannelHandler is used to indirectly process read-write events. Generally speaking, there are multiple steps in reading and writing. Pipeline is provided in Netty to organize these channelhandlers. Pipeline is a linked list container. You can add handlers at the beginning and end through addFirst and addLast.

The msg object in the method is the established connection pipe (NioSocketChannel)

channel.pipeline().addLast(new ChannelInboundHandlerAdapter(){
    
    // This method is used to handle new connections, accept and READ events
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        handlerAccept(work,msg);
    }

});

3. Register the new pipeline and initialize it

To get a new pipe (NioSocketChannel), you need to re register with EventLoop to receive messages. For simplicity, register directly in the same EventLoop as its parent Channel (you can also use a special EventLoop group to handle child pipeline events). Next, you also need to initialize the sub pipeline. Override the channelRead0 method to receive messages.

private void handlerAccept(NioEventLoopGroup group, Object msg) {
    
    NioSocketChannel channel= (NioSocketChannel) msg;
    
    EventLoop loop = group.next();
    
    loop.register(channel);
    
    channel.pipeline().addLast(new SimpleChannelInboundHandler<ByteBuf>() {
        
        @Override
        protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception {
            
            System.out.println(msg.toString(Charset.defaultCharset()));
            
        }
    });

}

NioServerSocketChannel and NioSocketChannel are parent-child relationships. Their pipeline s need to be registered and initialized respectively

ServerBootStrap usage

In the above cases, both the parent Channel and the child pipeline need to be initialized, which is more complex. Netty also provides a ServerBootStrap to further encapsulate the above operations such as registration, binding and initialization, which has simplified the call to Netty API.

Tell you the general usage of Netty through an Http service implementation. All its implementations are divided into the following steps

1. Initialization

Through ServerBootstrap, you can directly set the thread group of the service. The boss is used to handle the Accept event in nioseversocketchanl, and the Work group is used to handle IO read-write and asynchronous tasks submitted by users. Specify the Channel class to tell ServerBootstrap what kind of pipeline to maintain.

ServerBootstrap bootstrap = new ServerBootstrap();

EventLoopGroup boss = new NioEventLoopGroup(1);

EventLoopGroup work = new NioEventLoopGroup(8);

bootstrap.group(boss, work)
		//Specify the pipeline to be opened for automatic registration = = "nioserversocketchannel - >
        .channel(NioServerSocketChannel.class)
  1. Sets the Pipeline of the sub Pipeline

Then you can initialize the Pipeline of the sub Pipeline and bind the corresponding processing Handler for it. Our goal is to implement an Http service. The corresponding three basic operations are decoding, service processing and coding. Codec is a general processing of Htpp protocol. Netty has its own processor and can be added directly. Business processing needs to be written manually

// Initialize sub pipeline

bootstrap.childHandler(new ChannelInitializer<Channel>() {

            @Override

            protected void initChannel(Channel ch) {

                ch.pipeline().addLast("decode", new HttpRequestDecoder()); // input

                ch.pipeline().addLast("servlet", new MyServlet());

                ch.pipeline().addFirst("encode", new HttpResponseEncoder());// Output stream

            }

});

2. Business processing

By implementing SimpleChannelInboundHandler, you can directly handle the read event to receive the client's request. Next, you can construct a Response and set the status code, Response header and Response message to complete a simple Http service

private class MyServlet extends SimpleChannelInboundHandler {

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {

            FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_0, HttpResponseStatus.OK);

            response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/html;charset=utf-8");

            response.content().writeBytes("Upload complete".getBytes());

            ChannelFuture future = ctx.writeAndFlush(response);

            future.addListener(ChannelFutureListener.CLOSE);

    }

}

3. Binding port

The last step is to bind the port by serverbootrap

ChannelFuture future = bootstrap.bind(port);

future.addListener(future1 -> System.out.println("Service started successfully"));

Introduction to Netty Channel

In java Native NIO operations, Channel is a very core component. It can be used to connect both ends of the transmission and provide transmission and read-write related operations, such as binding ports, establishing connections, reading and writing messages, and closing pipes. Only SelectableChannel can be registered to the selector, and the selector can listen for reading and writing, so as to realize the non blocking function.

Netty's Channel is encapsulated in the native foundation, that is, the original functions can also be implemented in netty. At the same time, Pipeline and asynchronous execution mechanism are introduced.

1. Pipeline will assemble several channelhandlers (pipeline processors) in series, and all active or passive events for the pipeline will be processed on pipeline.

2. Asynchronous mechanism refers to asynchronous IO in the native NIO Channel, which is an unsafe behavior. It will trigger blocking and even cause deadlock. Therefore, generally, operating channels in non IO threads will be encapsulated in the form of tasks and submitted to IO threads for execution. All of these have been encapsulated and implemented in Netty. It is safe to call Netty Channel asynchronously.

Different Channel implementations in JAVA Native NIO will have different functions, which is similar in Netty. Its different subclasses will contain the corresponding channels in native NIO. Common are as follows:

[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-bg2gw4nu-1637830678945) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211111174258641. PNG)]

Netty Channel basic routine

Next, we have explained the basic usage of Channel with niodatagram Channel as an example. Based on the following steps, you can build a basic UDP service based on Netty Channel.

  1. Build thread group for IO threads
  2. Create and register pipes
  3. Initialize the pipeline and add a Channel Handler processor to it
  4. Binding port
//1. Build thread group, i.e. IO thread
NioEventLoopGroup boss = new NioEventLoopGroup(1);

//2. Create and register pipes
NioDatagramChannel datagram = new NioDatagramChannel();

boss.register(datagram);

//3. Initialize the pipeline and add a Channel Handler processor to it
datagram.pipeline().addLast(new ChannelInboundHandlerAdapter() {

    public void channelRead(ChannelHandlerContext ctx, Object msg) {

        System.out.println(msg);

    }

});

//4. Bind the port. After binding, start the IO thread and open the service to the outside world
datagram.bind(new InetSocketAddress(8080));

System.in.read();// Prevent main thread from ending 

The binding port is an IO operation, so the actual execution will be submitted to the IO thread (NioEventLoop) in the form of a task. But it still calls the JAVA Native NIO binding method in the end. This part is embodied in the NioDatagramChannel.doBind() method.

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-lyl2ocsb-1637830678946) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211125141401013. PNG)]

The Bootstrap.bind() binding method is also through channel.bind

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-udyshf3q-1637830678946) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211125141221192. PNG)]

And so on, the Channel.write() operation is invoked and the doWrite() is invoked, and finally transferred to Write or send in java nio Channel.

There are many similar methods:

bind==>doBind ==> java.nio.channels.Channel.bind()

write==>doWrite==> ...

connect==>doConnect==>...

close==>doClose ==>...

disconnect==>doDisconnect ==>.....

read==>doReadBytes ==> .....

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-a1aqiza1-1637830678947) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211125141837438. PNG)]

It can be seen that the doXXX method is an IO call directly to the native Channel. This is an Unsafe form when called by non IO threads, so all methods starting with do are not open (protected). Fortunately, Netty also provides an Unsafe in the Channel, which can directly call these methods.

Unsafe

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-juxkwjsn-1637830678947) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211125142113781. PNG)]

Unsafe is an internal class in the Channel, which can be used to directly operate IO methods in the Channel. Without going through asynchrony and pipeline. So calling the IO method in Unsafe will return immediately. However, like its name, this is not an unsafe form. The caller needs to ensure that the current call to unsafe is under the IO thread, otherwise an exception will be reported. Except for the following methods:

localAddress()

remoteAddress()

closeForcibly()

//This is an asynchronous method. Instead of returning immediately, it notifies ChannelPromise when it is completed

register(EventLoop, ChannelPromise)

deregister(ChannelPromise)

voidPromise()

Unsafe is an internal class of channels. Different channels correspond to different unsafe and provide different functions. For example, its structure and inheritance relationship are as follows:

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-wcrvtvhw-1637830678948) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211125142014614. PNG)]

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-kqsr39vl-1637830678948) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211112110502784. PNG)]

In addition, it should be noted that Unsafe not only forwards calls to Cahnnel as an intermediary, but also provides the following functions:

  1. Thread detection: is the current call IO thread?
  2. Status detection: judge whether it has been registered before writing
  3. Write cache: data is written to the temporary cache during write, and is actually committed only when flush
  4. Trigger read: EventLoop will notify unsafe based on the read event and send it to pipeline after being read by unsafe
  5. Therefore, the core role of Unsafe is not to call developers, but to call its internal components. It acts as a bridge between the three components of Channel, Eventloop and Pipeline.

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-2h9yzp4n-1637830678949) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211112103753777. PNG)]

For example, in a secondary reading scenario, the process is as follows:

  1. EventLoop triggers reading and notifies unsafe // unsafe.read()
  2. unsafe call channel to read messages / / channel.doReadMessages(ByteBuf)
  3. unsafe pass the message into pipeline (pipeline triggers message inbound) / / pipeline.fireChannelRead(msg)
private final class NioMessageUnsafe extends AbstractNioUnsafe {

        private final List<Object> readBuf = new ArrayList<Object>();

        @Override
        public void read() {
            assert eventLoop().inEventLoop();
            final ChannelConfig config = config();
            final ChannelPipeline pipeline = pipeline();
            final RecvByteBufAllocator.Handle allocHandle = unsafe().recvBufAllocHandle();
            allocHandle.reset(config);

            boolean closed = false;
            Throwable exception = null;
            try {
                try {
                    do {
                        // Call the read message of channel
                        int localRead = doReadMessages(readBuf);
                        if (localRead == 0) {
                            break;
                        }
                        if (localRead < 0) {
                            closed = true;
                            break;
                        }

                        allocHandle.incMessagesRead(localRead);
                    } while (allocHandle.continueReading());
                } catch (Throwable t) {
                    exception = t;
                }

                int size = readBuf.size();
                for (int i = 0; i < size; i ++) {
                    readPending = false;
                    // unsafe pass message into pipeline
                    pipeline.fireChannelRead(readBuf.get(i));
                }
                readBuf.clear();
                allocHandle.readComplete();
                pipeline.fireChannelReadComplete();

                if (exception != null) {
                    closed = closeOnReadError(exception);

                    pipeline.fireExceptionCaught(exception);
                }

                if (closed) {
                    inputShutdown = true;
                    if (isOpen()) {
                        close(voidPromise());
                    }
                }
            } finally {
                // Check if there is a readPending which was not processed yet.
                // This could be for two reasons:
                // * The user called Channel.read() or ChannelHandlerContext.read() in channelRead(...) method
                // * The user called Channel.read() or ChannelHandlerContext.read() in channelReadComplete(...) method
                //
                // See https://github.com/netty/netty/issues/2254
                if (!readPending && !config.isAutoRead()) {
                    removeReadOp();
                }
            }
        }
    }

Write process:

  1. Business development calls channel to write message / / channel.write(msg)
  2. channel writes messages to pipeline // pipeline.write(msg)
  3. Handler in pipeline handles messages asynchronously / / ChannelOutboundHandler.write()
  4. pipeline calls unsafe to write message / / unsafe.write(msg);
  5. unsafe call Channel to finish writing / / channel.doWrite(msg)

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ssyvt2cf-1637830678949) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211125144605877. PNG)]

private final class NioMessageUnsafe extends AbstractNioUnsafe {

	@Override
    protected void doWrite(ChannelOutboundBuffer in) throws Exception {
        final SelectionKey key = selectionKey();
        final int interestOps = key.interestOps();

        for (;;) {
            Object msg = in.current();
            if (msg == null) {
                // Wrote all messages.
                if ((interestOps & SelectionKey.OP_WRITE) != 0) {
                    key.interestOps(interestOps & ~SelectionKey.OP_WRITE);
                }
                break;
            }
            try {
                boolean done = false;
                for (int i = config().getWriteSpinCount() - 1; i >= 0; i--) {
                    if (doWriteMessage(msg, in)) {
                        done = true;
                        break;
                    }
                }

                if (done) {
                    in.remove();
                } else {
                    // Did not write all messages.
                    if ((interestOps & SelectionKey.OP_WRITE) == 0) {
                        key.interestOps(interestOps | SelectionKey.OP_WRITE);
                    }
                    break;
                }
            } catch (IOException e) {
                if (continueOnWriteError()) {
                    in.remove(e);
                } else {
                    throw e;
                }
            }
        }
    }

}

[external link image transfer failed. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-beybfcg9-1637830678950) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211125144920771. PNG)]

ChannelPipeline

There will be a unique pipeline in each pipeline, which is used to process events in the Channel in the way of flow, such as registration, binding ports, reading and writing messages, etc. These events will be rotated and processed in turn at each node in the pipeline flow, and each node can process the corresponding functions. This is a responsibility chain design pattern, which aims to enable each node to handle the focused business

[the external link image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-ntixirz1-1637830678950) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211112113848960. PNG)]

pipeline structure

How do events rotate in the pipeline? It adopts a two-way linked list structure, wraps a unique Handler through ChannelHandlerContext, and links the Context above and below the node through prev and next attributes to form a chain.

There are two contexts in the pipeline, Head and tail, corresponding to the beginning and end of the chain.

[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-jea72am6-1637830678951) (C:% 5cusers% 5c75412% 5cappdata% 5croaming% 5ctypora user images% 5cimage-20211112140554845. PNG)]

ChannelHandler

ChannelHandler refers to the nodes in pipeline. There are three types:

  1. Inbound processor: the implementation of ChannelInboundHandler, which can be used to handle inbound events such as message reading
  2. Outbound processor: the implementation of ChannelOutboundHandler, which can be used to handle message writing, port binding, inbound and outbound events
  3. Inbound and outbound processor: the implementation of ChannelDuplexHandler, which can handle all inbound and outbound events. If you want to write some codec operations in a class, you can use the processor to implement them.

Inbound and outbound events

Events in Channel can be divided into outbound and inbound.

Inbound event: refers to events that occur in the station, such as read message processing, pipeline registration, pipeline activation (bound port or connected), which are initiated passively by EventLoop based on IO events. Note that all inbound event triggers must be executed by a subclass of ChannelInBoundInvoker.

Outbound event: an outbound event refers to a request or a message written to the other end (outbound) of the Channel. Such as bind, connect, close, write, flush, etc. They are triggered by ChannelOutboundInvoker and processed by ChannelOutboundHandler. Unlike inbound events, they are initiated by the developer himself.

Event triggering

As can be seen from the figure below, pipeline and Context implement inbound and outbound interfaces respectively, indicating that they can trigger all inbound and outbound events, while Channel only inherits outbound ports and can only trigger outbound events.

[the external link image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-eb4ugsyv-1637830678952) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211112142400178. PNG)]

ChannelHandlerContext

The main functions of Context are as follows:

  1. Link up and down nodes on structure

  2. Pass inbound and outbound events. All events can be passed up and down by Context

  3. Ensure that the processing is on the IO thread. All IO operations mentioned above need to be submitted to the IO thread asynchronously for processing. This logic is implemented by Context. For example, the following binding operations ensure the execution of IO threads:

Chain processing flow (event transmission)

Inbound and outbound events are initiated by Channel or pipeline and transmitted up and down by Context. If it is an inbound event, it will be passed down from the head to the tail and skip the OutboundHandler, while the outbound event will be passed up from the tail and skip the InboundHandler processor.

Next, the processing process of pipeline is illustrated by reading and writing two examples.

Read pipeline messages

1. Initial process

NioDatagramChannel channel = new NioDatagramChannel();

new NioEventLoopGroup().register(channel);

Thread.sleep(100);

channel.bind(new InetSocketAddress(8081));

2. Add inbound processing node 1 to the pipeline

pipeline.addLast(new ChannelInboundHandlerAdapter(){

    public void channelRead(ChannelHandlerContext ctx, Object msg) {

        String message = (String) msg;

        System.out.println("Inbound event 1:"+msg);

        message+=" Processed";

        ctx.fireChannelRead(message);

    }

});

3. Add outbound processing node 1 to the pipeline

//Outbound processing

pipeline.addLast(new ChannelOutboundHandlerAdapter(){

    public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {

        System.out.println("Outbound event 1:"+msg);

    }

});

4. Add inbound processing node 2 to the pipeline

        pipeline.addLast(new ChannelInboundHandlerAdapter(){
            public void channelRead(ChannelHandlerContext ctx, Object msg) {
                String message = (String) msg;
                System.out.println("Inbound event 2:"+msg);
                message+=" Processed";
                ctx.writeAndFlush(message);
            }

        });

5. Manually trigger inbound events

pipeline.fireChannelRead("hello luban");

5. After implementation, the following results were obtained

Inbound event 1: hello luban
 Inbound event 2: hello luban Processed
 Outbound event 1: hello luban Processed processed

Description of execution process:

  1. The inbound processing is triggered based on the pipeline, which is first processed by the header and passed down
  2. Node 1 receives the message and rewrites the message through ctx.fireChannelRead(); Pass down
  3. Node 2 receives the message and prints it. At this time, Section 2 does not call ctx.fireChannelRead(); Therefore, the processing flow will not be transferred to the tail node
  4. Processing flow.

8.TCP packet sticking / unpacking

TCP is a "flow" protocol. The so-called flow is a string of data without boundaries. You can think of the water in the river, which is connected, and there is no boundary between them. The bottom layer of TCP does not understand the specific meaning of business data. It will divide packets according to the actual situation of TCP buffer. Therefore, in terms of business, it believes that a complete packet may be split into multiple packets for transmission by TCP, or multiple small packets may be encapsulated into a large packet for data transmission. This is the so-called TCP packet sticking and unpacking problem.

TCP packet sticking / unpacking problem description

We illustrate TCP packet sticking and unpacking by illustration:

[external link image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-txprwkfd-1637830678953) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211109154919713. PNG)]

Suppose the client sends two data packets D1 and D2 to the server respectively. Since the number of bytes read by the server at one time is uncertain, the above four situations may exist.

(1) The server reads two independent data packets, D1 and D2, without sticking and unpacking;

(2) The server receives two data at a time, D1 and D2 are stuck together, which is called TCP sticky packet;

(3) The server reads two data packets twice, and reads the complete D1 packet and part of D2 packet for the first time_ 1. The remaining content D2 of D2 packet is read for the second time_ 2. This is called TCP unpacking;

(4) The server reads two data packets twice, and reads part of D1 packet for the first time_ 1. Read the remaining content D1 of D1 package for the second time_ 2 and complete D2 packet, which is also TCP unpacking;

If the TCP receiving sliding window of the server is very small and the data packets D1 and D2 are relatively large, the fifth situation is likely to occur, that is, the server can receive the D1 and D2 packets completely several times, and the packets are disassembled many times during this period.

Causes of TCP packet sticking / unpacking

There are three reasons for the problem, which are as follows:

(1) The byte size of the application write writer is larger than the size of the socket send buffer

(2) TCP segmentation for MSS size

(3) The payload of Ethernet frame is larger than MTU for IP fragmentation.

The diagram is as follows:

[the external chain image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-evgsqbjk-1637830678953) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211109160335391. PNG)]

Solution strategy of TCP packet sticking / unpacking

Since the underlying TCP cannot understand the business data of the upper layer, it is impossible to ensure that the data packets will not be split and reorganized at the lower layer. This problem can only be solved through the application protocol design of the upper layer. According to the solutions of the mainstream protocols in the industry, the following solutions can be summarized:

(1) The message length is fixed. For example, the size of each message is 200 bytes of fixed length. If it is not enough, fill in the space;

(2) Add a carriage return line feed character at the end of the package for segmentation, such as FTP protocol;

(3) The message is divided into message header and message body. The message header contains the total length of the message (or the length of the message body);

(4) More complex application layer protocols.

9. Several common codec classes provided by netty

In order to solve the problem of half packet reading and writing caused by TCP packet sticking / unpacking, Netty provides a variety of codecs by default to process half packets. (the implementation of handling TCP packet sticking / unpacking is generally implemented in the decoding class). Since TCP transmits data in the form of stream, the application protocol of the upper layer often adopts the following four methods in order to distinguish messages.

(1) When the message length is fixed and the total length of the message read is a fixed length LEN, it will be considered that a complete message has been read: set the counter and restart reading the next data message;

(2) Using carriage return and line feed as message terminator, such as FTP protocol, is widely used in text protocol

(3) The special separator is used as the sign of the end of the message, and the carriage return line feed is a special end separator;

(4) The total length of the message is identified by defining the length field in the message header

Netty makes a unified abstraction of the above four applications and provides four decoders to solve the corresponding problems, which is very convenient to use. With these decoders, users do not need to manually decode the read messages, nor do they need to consider the sticking and unpacking of TCP.

FixedLengthFrameDecoder

Fixed length frame decoder is also called fixed length decoder, which can automatically decode fixed length messages. Corresponding to the first method above

ch.pipeline().addLast(new LineBasedFrameDecoder(20));

The parameter represents the fixed length of the message

LineBasedFrameDecoder

The working principle of LineBasedFrameDecoder is to traverse the readable bytes in ByteBuf in turn to determine whether there is "\ n" or "\ r\n". If so, this position is the end position, and the bytes from the readable index to the end index position form a line. She is a decoder that ends with a newline character. It supports two decoding methods with or without terminator, and supports configuring the maximum length of the current line. If no newline character is found after the maximum length read continuously, an exception will be thrown and the previously read exception code stream will be ignored. Corresponding to the second method above

ch.pipeline().addLast(new LineBasedFrameDecoder(1024));

Parameter represents the maximum length of a single message

DelimiterBasedFrameDecoder

DelimiterBasedFrameDecoder, also known as separator decoder, can automatically decode messages with a specified separator as the end identifier of the code stream. Corresponding to the third method above

ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());
ch.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, delimiter));

The first parameter 1024 indicates the maximum length of a single message. When the separator is still not found after reaching this length, a TooLongFrameException exception will be thrown to prevent memory overflow due to the lack of separator in the abnormal code stream, which is the reliability protection of Netty's decoder; The second parameter is the separator buffer object.

LengthFieldBasedFrameDecoder

The LengthFieldBasedFrameDecoder is also called a custom length decoder. It is distinguished by the length of the message. It can easily solve the problem of reading half a packet by passing in the correct parameters. Corresponding to the third method above, it is also the most common method.

The LengthFieldBasedFrameDecoder is a custom length decoder, so the six parameters in the constructor are basically described around the defined length field.

  1. maxFrameLength - maximum length of data frame sent

  2. lengthFieldOffset - defines the subscript of the length field in the sent byte array. In other words: the place with the subscript ${lengthFieldOffset} in the sent byte array is the beginning of the length field

  3. lengthFieldLength - describes the length of the defined length field. In other words: when sending byte array bytes, the byte array bytes[lengthFieldOffset, lengthFieldOffset+lengthFieldLength] field corresponds to the defined length field part of the

  4. lengthAdjustment - satisfies the formula: sent byte array bytes.length - lengthFieldLength = bytes[lengthFieldOffset, lengthFieldOffset+lengthFieldLength] + lengthFieldOffset + lengthAdjustment

  5. initialBytesToStrip - received transmission packet, removing the first initialBytesToStrip bit

  6. failFast - true: if the read length field exceeds maxFrameLength, a TooLongFrameException will be thrown. false: TooLongFrameException will be thrown only after the bytes represented by the value of the length field are actually read. It is set to true by default. It is recommended not to modify it, otherwise it may cause memory overflow

For example, chestnuts:

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-su2jq4bu-1637830678954) (C:% 5cusers% 5c75412% 5cappdata% 5creading% 5ctypora% 5ctypora user images% 5cimage-20211109174417476. PNG)]

[the external link image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-vdepvupb-1637830678955) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211109175119161. PNG)]

According to the figure:

The second parameter is 1, the array position starts from 0, and the third parameter is 2. Since there is no data loss after decoding, the fourth parameter is 0, which is determined by the formula

Send packet length = value of length field + lengthFieldOffset + lengthFieldLength + lengthAdjustment.

The fourth parameter can be calculated as - 3. The first parameter here is set to the maximum value of int, which can be modified according to the business.

So the code is:

pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE, 1, 2, -3, 0));

Remember the formula: send packet length = value of length field + lengthFieldOffset + lengthFieldLength + lengthAdjustment.

Custom decoder

Although netty provides us with a decoder, here is a custom decoding example to understand how netty implements automatic decoding.

Here we implement decoding by inheriting ByteToMessageDecoder. We can easily implement our own decoder through ByteToMessageDecoder. (the four methods described above also inherit the ByteToMessageDecoder Implementation)

Similarly, we implement our own decoder by taking the example in the custom length decoder of LengthFieldBasedFrameDecoder

The code is as follows:

package com.gbcom.idic.rosterCollector.netty.server.decoder;

import com.gbcom.idic.rosterCollector.netty.common.constants.CubeMsgConstant;
import com.gbcom.idic.rosterCollector.netty.common.pojo.CubeMsg;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageDecoder;
import lombok.extern.slf4j.Slf4j;

import java.nio.charset.StandardCharsets;
import java.util.List;

/**
 * @Author: dinghao
 * @Date: 2020/10/22 15:11
 */
@Slf4j
public class TransforDecoder extends ByteToMessageDecoder {

    /**
     * devType	    1
     * length	    2
     * devId	    17
     * msgType	    2
     * seqence	    4
     * data	        N
     */

    //Minimum data length: the first standard bit is 1 byte
    private static int MIN_DATA_LEN = CubeMsgConstant.devTypeLength + CubeMsgConstant.totalLength + CubeMsgConstant.devIdLength + CubeMsgConstant.msgTypeLength + CubeMsgConstant.seqenceLength;

    @Override
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
        int readableBytes = byteBuf.readableBytes();
        if (readableBytes >= MIN_DATA_LEN) {
            log.debug("Start decoding data");
            //Pointer to mark read operation
            byteBuf.markReaderIndex();
            // Read device type
            byte devType = byteBuf.readByte();
            // Read message length (message header + message body)
            short length = byteBuf.readShort();
            if (length > readableBytes) {
                log.debug("Insufficient data length, data protocol len Length:{},The actual readable contents of the data package are:{}Waiting to process unpacking", length, readableBytes);
                byteBuf.resetReaderIndex();
                /*
                 **End decoding. This indicates that the data is not complete. out and in will be judged in the callDecode of the parent ByteToMessageDecoder
                 * If there is still readable content in in, i.e. in.isReadable is true, the content in the calculation will be retained until the next data arrival. The data of the two frames will be combined and decoded.
                 * So as to solve the unpacking problem
                 */
                return;
            }
            byte[] devIdData = new byte[CubeMsgConstant.devIdLength];
            byteBuf.readBytes(devIdData);
            String devId = new String(devIdData, StandardCharsets.UTF_8);
            short msgType = byteBuf.readShort();
            int seqence = byteBuf.readInt();
            byte[] data = new byte[length - MIN_DATA_LEN];
            byteBuf.readBytes(data);
            CubeMsg msg = new CubeMsg();
            msg.setDevType(devType);
            msg.setLength(length);
            msg.setDevId(devId);
            msg.setMsgType(msgType);
            msg.setSeqence(seqence);
            msg.setData(data);
            list.add(msg);
            if (length < readableBytes) {
                //If out has a value and in is still readable, it will continue to call the decode method to decode the content in again, so as to solve the sticking problem
                log.debug("Data length is too long, data protocol len Length:{},The actual readable contents of the data package are:{}Waiting to process sticky package", length, readableBytes);
            }
        } else {
            log.debug("The data length does not meet the requirements. The expected minimum length is:" + MIN_DATA_LEN + " byte");
            return;
        }
    }
}

ByteToMessageDecoder source code analysis:

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        if (msg instanceof ByteBuf) {
            CodecOutputList out = CodecOutputList.newInstance();
            try {
                ByteBuf data = (ByteBuf) msg;
                first = cumulation == null;
                if (first) {
                    cumulation = data;
                } else {
                    cumulation = cumulator.cumulate(ctx.alloc(), cumulation, data);
                }
                callDecode(ctx, cumulation, out);
            } catch (DecoderException e) {
                throw e;
            } catch (Throwable t) {
                throw new DecoderException(t);
            } finally {
                if (cumulation != null && !cumulation.isReadable()) {
                    numReads = 0;
                    cumulation.release();
                    cumulation = null;
                } else if (++ numReads >= discardAfterReads) {
                    // We did enough reads already try to discard some bytes so we not risk to see a OOME.
                    // See https://github.com/netty/netty/issues/4275
                    numReads = 0;
                    discardSomeReadBytes();
                }

                int size = out.size();
                decodeWasNull = !out.insertSinceRecycled();
                fireChannelRead(ctx, out, size);
                out.recycle();
            }
        } else {
            ctx.fireChannelRead(msg);
        }
    }
    /**
     * Called once data should be decoded from the given {@link ByteBuf}. This method will call
     * {@link #decode(ChannelHandlerContext, ByteBuf, List)} as long as decoding should take place.
     *
     * @param ctx           the {@link ChannelHandlerContext} which this {@link ByteToMessageDecoder} belongs to
     * @param in            the {@link ByteBuf} from which to read data
     * @param out           the {@link List} to which decoded messages should be added
     */
    protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {
        try {
            while (in.isReadable()) {
                int outSize = out.size();

                if (outSize > 0) {
                    fireChannelRead(ctx, out, outSize);
                    out.clear();

                    // Check if this handler was removed before continuing with decoding.
                    // If it was removed, it is not safe to continue to operate on the buffer.
                    //
                    // See:
                    // - https://github.com/netty/netty/issues/4635
                    if (ctx.isRemoved()) {
                        break;
                    }
                    outSize = 0;
                }

                int oldInputLength = in.readableBytes();
                decode(ctx, in, out);

                // Check if this handler was removed before continuing the loop.
                // If it was removed, it is not safe to continue to operate on the buffer.
                //
                // See https://github.com/netty/netty/issues/1664
                if (ctx.isRemoved()) {
                    break;
                }

                if (outSize == out.size()) {
                    if (oldInputLength == in.readableBytes()) {
                        break;
                    } else {
                        continue;
                    }
                }

                if (oldInputLength == in.readableBytes()) {
                    throw new DecoderException(
                            StringUtil.simpleClassName(getClass()) +
                            ".decode() did not read anything but decoded a message.");
                }

                if (isSingleDecode()) {
                    break;
                }
            }
        } catch (DecoderException e) {
            throw e;
        } catch (Throwable cause) {
            throw new DecoderException(cause);
        }
    }

10. Implementation of private protocol based on Netty

Let's use the above netty related knowledge to implement the private protocol based on netty. When customizing the protocol, ensure the reliability, security and scalability of the protocol design.

reliability design

Netty protocol may run in a very bad network environment, and network timeout, flash off, dead process or slow processing of the other party may occur. In order to ensure that netty protocol can still work normally or recover automatically in these extreme abnormal scenarios, its reliability needs to be planned and designed uniformly.

  1. heartbeat mechanism

    The heartbeat mechanism is used to detect the interoperability of the link. Once a network fault is found, the link is closed immediately and reconnected actively

  2. Reconnection mechanism

    It is generally used together with the heartbeat mechanism. When the link is closed, it is reconnected at a certain interval

  3. Duplicate login protection

    When the client is connected successfully, the link is in the normal state, and the client is not allowed to log in repeatedly to prevent repeated reconnection in the abnormal state of the client, resulting in the depletion of handle resources

  4. Message cache retransmission

    Whether the client or the server, after the link terminal and before the link recovery, the data to be sent cached in the message queue cannot be lost. After the link recovery, resend these messages to ensure that the messages are not lost during the link interruption. Considering the memory overflow problem, it is recommended to set an upper limit for message queue. When the upper limit is reached, you should refuse to add new messages to the queue or clear old messages

Safety design

In order to ensure the security of the server, the server needs to verify the legitimacy of the IP address of the client. If it is in the white list, the verification passes; Otherwise, reject the other party's connection

Extensible design

The agreement needs to have a certain expansion ability

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-crpqudn5-1637830678956) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211122203634019. PNG)]

Code implementation:

11.Netty implements its own intranet penetration tool

This section implements a simple intranet penetration tool through Netty. Its principle is as follows:

The server is deployed in the external network environment, and the server is a tcp service that opens port 7777

The client is configured with the local port 80 (which can be changed freely here), the external access port 10000 (which can be changed freely here) and the port 7777 connecting to the server (which is agreed between the client and the server)

1. When the server starts, the tcp service that opens port 7777 starts to receive the connection from the client

2. When the client starts, connect to port 7777 of the server to establish a tcp connection. After the connection is successful, send the account and password to the server

3. After the server verifies the account and password successfully, it establishes a tcp service that the client needs to access the external port (10000 as an example) at the server, and returns whether the connection between the client and the service is successful. The failed server will close the connection with the client

4. The client determines whether to close the program according to the returned information. At this time, the external can directly access the port accessed by the client through the server

5. Browser access 10000 ports

6. The connection between the browser and the 10000 port of the server is successful. At this time, the server sends a connection request message carrying the id of the connection channel established between the browser and the 10000 port of the server to the client through the channel of the port connected with the client (the connection established through port 7777)

7. After the client receives the request connection with the id, the client establishes a connection with the local service (80 as an example), and puts the connection between the id and the client and the local service into the Map.

8. After the browser successfully establishes a connection with the 10000 port of the server, the server sends a data message carrying the id of the connection channel established between the browser and the 10000 port of the server and the request data to the client through the channel of the port connected with the client (the connection established through port 7777)

9. The client receives the id and request data (the request data of the browser), and forwards the request data to the channel between the client and the local service (id acquisition)

10. The client reads the response of the local service and returns the response result and id to the server through the channel of the port connected with the client (through the connection established through port 7777)

11. The server reads the response, obtains the connection between the server and the browser according to the id, and writes the response to the browser

[the external chain image transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-ymrdeji2-1637830678957) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211029162433745. PNG)]

[the external chain picture transfer fails, and the source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-usyeo7qi-1637830678958) (C:% 5cusers% 5c75412% 5cappdata% 5cloading% 5ctypora% 5ctypora user images% 5cimage-20211029174003598. PNG)]

Code implementation:

12.Netty source code analysis

Follow up supplement

    break;
                } else {
                    continue;
                }
            }

            if (oldInputLength == in.readableBytes()) {
                throw new DecoderException(
                        StringUtil.simpleClassName(getClass()) +
                        ".decode() did not read anything but decoded a message.");
            }

            if (isSingleDecode()) {
                break;
            }
        }
    } catch (DecoderException e) {
        throw e;
    } catch (Throwable cause) {
        throw new DecoderException(cause);
    }
}


## 10. Implementation of private protocol based on Netty

​	Let's use the above introduction netty Based on relevant knowledge Netty Private agreement. When customizing the protocol, ensure the reliability, security and scalability of the protocol design.



### reliability design 

​	Netty The protocol may run in a very bad network environment. Network timeout, flash off, dead process or slow processing of the other party may occur. In order to ensure that in these extremely abnormal scenarios Netty The protocol can still work normally or recover automatically, so its reliability needs to be planned and designed uniformly.

1. heartbeat mechanism 

   The heartbeat mechanism is used to detect the interoperability of the link. Once a network fault is found, the link is closed immediately and reconnected actively

2. Reconnection mechanism

   It is generally used together with the heartbeat mechanism. When the link is closed, it is reconnected at a certain interval

3. Duplicate login protection

   When the client is connected successfully, the link is in the normal state, and the client is not allowed to log in repeatedly to prevent repeated reconnection in the abnormal state of the client, resulting in the depletion of handle resources

4. Message cache retransmission

    Whether the client or the server, after the link terminal and before the link recovery, the data to be sent cached in the message queue cannot be lost. After the link recovery, resend these messages to ensure that the messages are not lost during the link interruption. Considering the memory overflow problem, it is recommended to set an upper limit for message queue. When the upper limit is reached, you should refuse to add new messages to the queue or clear old messages

### Safety design

​	In order to ensure the security of the server, the server needs to support the client IP Verify the validity of the address. If it is in the white list, it passes the inspection; Otherwise, reject the other party's connection

### Extensible design

​	The agreement needs to have a certain expansion ability



[External chain picture transfer...(img-CrpQUDn5-1637830678956)]



Code implementation:





## 11.Netty implements its own intranet penetration tool

This section was adopted Netty Implement a simple intranet penetration tool. Its principle is as follows:

The server is deployed in the external network environment, and the server is open to port 7777 tcp service

The client is configured with the local port 80 (which can be changed freely here), the external access port 10000 (which can be changed freely here) and the port 7777 connecting to the server (which is agreed between the client and the server)

1,When the server starts, open the port 7777 tcp The service starts and starts receiving client connections

2,When the client starts, the 7777 port connecting to the server is established tcp After the connection is successful, send the account and password to the server

3,After the server successfully verifies the account and password, establish the external access port (10000 as an example) required by the client on the server tcp Service, and returns whether the connection between the client and the service is successful. If it fails, the server will close the connection with the client

4,The client determines whether to close the program according to the returned information. At this time, the external can directly access the port accessed by the client through the server

5,Browser access 10000 ports

6,The connection between the browser and the 10000 port of the server is successful. At this time, the server sends a message carrying the connection channel between the browser and the 10000 port of the server to the client through the channel of the port connected with the client (the connection established through port 7777) id Request connection message for

7,Client receiving and carrying id After the request is connected, the client establishes a connection with the local service (80 as an example), and id Connections to clients and local services Map Yes.

8,After the browser successfully establishes a connection with the 10000 port of the server, the server again sends a message to the client through the channel of the port connected with the client (the connection established through port 7777) carrying the connection channel established between the browser and the 10000 port of the server id And data messages requesting data

9,Client received id And request data (request data of the browser) to forward the request data to the channel of the client and the local service( id Get)

10,The client reads the response of the local service and compares the result of the response with the id The server returns to the server through the channel of the port connected with the client (the connection established through port 7777)

11,The server reads the response according to id Get the connection between the server and the browser, and write the response to the browser





[External chain picture transfer...(img-YMrdeJi2-1637830678957)]



[External chain picture transfer...(img-uSyEo7qi-1637830678958)]

Code implementation:





## 12.Netty source code analysis

Follow up supplement



Tags: Netty

Posted on Fri, 26 Nov 2021 00:02:14 -0500 by whitsey