Netty principle: what does ByteBuffer do to Nio bytebuffer to improve efficiency?

ByteBuf

Disadvantages of ByteBuffer in NIO:

A is fixed in length, unable to expand and shrink dynamically, and lacks flexibility B uses a position to record the read-write index position. When the read-write mode is switched, the flip method needs to be called manually, which increases the complexity of use. C has limited functions and often needs to be encapsulated by itself in the process of use

1) Classification

According to the location of memory, it is divided into heap buffer, direct buffer and composite buffer.

A heap buffer

The data is stored in the heap space of the JVM, and the byte array byte [] is actually used for storage. Advantages: data can be created and released quickly, and the internal array can be accessed directly Disadvantages: when reading and writing data, you need to copy the data to the direct buffer for network transmission.

B direct buffer

Not in the heap, but using the local memory of the operating system. Advantages: in the process of data transmission using Socket, one copy is reduced and the performance is higher. Disadvantages: the space released and allocated is more expensive and needs to be used more carefully.

C composite buffer

Merge two or more buffers with different memory Advantages: it can be operated uniformly

Application scenario: when the communication thread uses the buffer, it often uses the direct buffer, while when the business message uses the buffer, it often uses the heap buffer. When the http package and the request header + request body have different characteristics and choose different locations for storage, the two can be spliced

You can see the type through iterator traversal

  public static void main(String[] args) { 
//    Heap memory other ways
        ByteBuf byteBuf = Unpooled.buffer();
        //Direct memory buffer
        ByteBuf dBuf = Unpooled.directBuffer();
        //This type is compound buffer. Heap memory and direct memory can be combined as parameters
        CompositeByteBuf csbuf = Unpooled.compositeBuffer();
        csbuf.addComponents(byteBuf, dBuf);
        //    This buffer can be traversed with an iterator
        Iterator<ByteBuf> iterator = csbuf.iterator();
        while (iterator.hasNext()) {
            System.out.println(iterator.next());
        }
  }

You can see that the traversal here is the heap memory buffer and direct memory buffer we put in successively

The space and release of direct memory buffer are relatively complex, and the speed is slightly slower than heap memory. How can we solve this problem?

D. concept of pooling

netty solves the complexity and efficiency of memory space allocation and release through memory pool. Memory pool, ByteBuf can be recycled to improve utilization. However, management and maintenance are complex.

Unpooled is the tool class for non pooled buffers.

The main difference is that pooled memory is managed by netty and non pooled memory is recycled by GC.

E. recycling method

The recycling method is reference counting. The specific rule is to judge whether the current object will still be used by recording the number of references. When the object is called, the reference count is + 1. When the object is released, the reference count is - 1. When the number of references is 0, the object can be recycled.

Disadvantages: may cause memory leaks. When the object is unreachable, the JVM will recycle it through GC, but the reference count may not be 0. The object cannot be returned to the memory pool, resulting in memory leakage. netty can only be checked by sampling the memory buffer.

Code effect of GC recycling method

public static void main(String[] args) {
        ByteBuf buf = Unpooled.buffer(10);
        System.out.println(buf);
        //Value of reference count
        System.out.println(buf.refCnt());
        //Hold means to count + 1
        buf.retain();
        System.out.println(buf.refCnt());
        //Release meaning count - 1
        buf.release();
        System.out.println(buf.refCnt());

    }

2) Working principle

Different from ByteBuffer, a pointer is added to record the index position in read mode and write mode through two pointers. The read pointer is called readerIndex and the write pointer is called writerIndex.

A read write separation

When the clear() method is executed, the index position is cleared back to its original position, but the data remains unchanged.

The mark and reset methods are also applicable in ByteBuf, such as markReaderIndex and resetReaderIndex.

Use code to verify the effect

Take a look at the initial value first. The following code is in the main method.

    public static void main(String[] args) {
        ByteBuf buf = Unpooled.buffer();
        //The default initialization is 256
        System.out.println(buf.capacity());
        //View read / write index
        System.out.println(buf.readerIndex());
        System.out.println(buf.writerIndex());
        //Case sensitive
        System.out.println(buf.writableBytes());
    }

Different parameters have different methods to show

Next, write data

		buf.writeBytes("hello index".getBytes());
        //The default initialization is 256
        System.out.println(buf.capacity());
        //View read / write index
        System.out.println(buf.readerIndex());
        System.out.println(buf.writerIndex());
        //Case sensitive
        System.out.println(buf.writableBytes());

Read data

        System.out.println("--------Only 5 bytes are read hello");
        for (int i = 0; i < 5; i++) {
            System.out.print((char) buf.readByte());

        }
        System.out.println();
        //The default initialization is 256
        System.out.println(buf.capacity());
        //View read / write index
        System.out.println(buf.readerIndex());
        System.out.println(buf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + buf.writableBytes());
        System.out.println("Read / Write Bytes:" + buf.readableBytes());

Recyclable area code verification

After reading, the five bytes we have read become recyclable areas, and we can call methods to recycle them

After recycling, the writeable position of the method will become the current size + recycling size

  		//Reclaim disposable space
        buf.discardReadBytes();
        System.out.println("-------Recycling waste space");
        //The default initialization is 256
        System.out.println(buf.capacity());
        //View read / write index
        System.out.println(buf.readerIndex());
        System.out.println(buf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + buf.writableBytes());
        System.out.println("Read / Write Bytes:" + buf.readableBytes());

We can see that our read index zeroed and writable area has become the original and recyclable area

Mark rollback is the same as the previous charbuffer, except here are two indexes

   System.out.println("--------read index And back off");
        //Both read and write indexes apply here
        buf.markReaderIndex();
        //buf.markWriterIndex();
        int end = buf.writerIndex();
        for (int i = buf.readerIndex(); i < end; i++) {
            System.out.print((char) buf.readByte());

        }
        System.out.println();
        //Back to mark
        buf.resetReaderIndex();
        //The default initialization is 256
        System.out.println(buf.capacity());
        //View read / write index
        System.out.println(buf.readerIndex());
        System.out.println(buf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + buf.writableBytes());
        System.out.println("Read / Write Bytes:" + buf.readableBytes());
B deep and shallow copy

Shallow copy refers to a reference to an object and does not create a new object. The new object and the original object affect each other.

Implementation of shallow copy code verification

  1. The shallow copy method proved to be a reference
  2. The slice copy is read only and the non writable interval is readerindex - writerindex
    public static void main(String[] args) {
        ByteBuf buf = Unpooled.buffer();
        buf.writeBytes("hello bytebuf copy".getBytes());
        System.out.println("capacity:" + buf.capacity());
        //View read / write index
        System.out.println("readerIndex:" + buf.readerIndex());
        System.out.println("writerIndex:" + buf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + buf.writableBytes());
        System.out.println("Read / Write Bytes:" + buf.readableBytes());

        //Copy shallow copy
        ByteBuf newbuf = buf.duplicate();
        System.out.println("-------------duplicate newbuf");

        System.out.println("capacity:" + newbuf.capacity());
        //View read / write index
        System.out.println("readerIndex:" + newbuf.readerIndex());
        System.out.println("writerIndex:" + newbuf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + newbuf.writableBytes());
        System.out.println("Read / Write Bytes:" + newbuf.readableBytes());
        //Write new data in new buf
        System.out.println("-------------duplicate newbuf add data");
        newbuf.writeBytes("from newbuf".getBytes());

        //Then read the two buf s to see the difference
        //If the read size exceeds the case index, an error will be reported. We need to set it
        buf.writerIndex(30);
        for (int i = 0; i < 13; i++) {
            System.out.print((char) buf.readByte());
        }
        System.out.println();
        System.out.println("capacity:" + buf.capacity());
        //View read / write index
        System.out.println("readerIndex:" + buf.readerIndex());
        System.out.println("writerIndex:" + buf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + buf.writableBytes());
        System.out.println("Read / Write Bytes:" + buf.readableBytes());

        //Sometimes we only need to get the most important part, and the unread part netty also provides us with methods
        //slice() is the area between the shallow copy section readerindex - writerindex
        //The capacity of this read only and non writable slice is the size of the original read area
        ByteBuf sliceBuf = buf.slice();
        //Writing data causes an exception
        System.out.println("---------sliceBuf");
        System.out.println("capacity:" + sliceBuf.capacity());
        //View read / write index
        System.out.println("readerIndex:" + sliceBuf.readerIndex());
        System.out.println("writerIndex:" + sliceBuf.writerIndex());
        //Case sensitive
        System.out.println("Writable area:" + sliceBuf.writableBytes());
        System.out.println("Read / Write Bytes:" + sliceBuf.readableBytes());
    }

Effect output analysis

Deep copy, which copies the entire object, is completely independent of the original object.

// Deep replication
ByteBuf copyBuf = buf.copy();
System.out.println("------copyBuf");
System.out.println("capacity:" + copyBuf.capacity());
//View read / write index
System.out.println("readerIndex:" + copyBuf.readerIndex());
System.out.println("writerIndex:" + copyBuf.writerIndex());
//Case sensitive
System.out.println("Writable area:" + copyBuf.writableBytes());
System.out.println("Read / Write Bytes:" + copyBuf.readableBytes());

System.out.println("------- add data copybuf");
copyBuf.writeBytes("from copyBuf".getBytes());

copyBuf.writerIndex(43);
for (int i = copyBuf.readerIndex(); i < 43; i++) {
    System.out.print((char) copyBuf.readByte());
}
System.out.println();
System.out.println("----------primary buf");
buf.writerIndex(43);
for (int i = buf.readerIndex(); i < 43; i++) {
    System.out.print((char) buf.readByte());
}
System.out.println();

Analysis results

Summary

duplicate and slice methods to achieve all shallow copies and some shallow copies. Copy, part deep copy, part represents readable space.

3) Capacity expansion mechanism

Storage of A ByteBuffer

When put ting data, ByteBuffer will check whether the remaining space is insufficient. If it is insufficient, an exception will be thrown.

ByteBuffer buffer = ByteBuffer.allocate(8);
buffer.put("yu".getBytes());

----------------------------------------------------

    public final ByteBuffer put(byte[] src) {
        return put(src, 0, src.length);
    }
    
    // Additional receive offset (starting position of stored data) and data length
    public ByteBuffer put(byte[] src, int offset, int length) {
        // Verify the validity of parameters
        checkBounds(offset, length, src.length);
        // If the len gt h of the data to be stored > the remaining free space, the buffer out of bounds exception is thrown
        if (length > remaining())
            throw new BufferOverflowException();
        // If there is enough space left, calculate the end position of storage = offset + data length    
        int end = offset + length;
        for (int i = offset; i < end; i++)
            this.put(src[i]);
        return this;
    }    

If you want to manually expand the capacity of ByteBuffer, you can verify whether the remaining space is sufficient before put ting. If not, create a new ByteBuffer, ensure that the new capacity is sufficient, copy the old buffer data to the new buffer, and then continue to store data.

Storage and expansion of B ByteBuf

When writing data, first judge whether to expand the capacity. If the current space is small (< 4M), multiply 64 as the cardinality (10 - > 64 - > 128 - > 256). If the current space is large (> 4M), increase 4M each time. This method is called "stepping".

Verify capacity expansion

    public static void main(String[] args) {
        ByteBuf buf = Unpooled.buffer(10);
        System.out.println("capacity:" + buf.capacity());
        for (int i = 0; i < 11; i++) {
            buf.writeByte(i);
        }
        System.out.println("capacity:" + buf.capacity());
        for (int i = 0; i < 65; i++) {
            buf.writeByte(i);
        }
        System.out.println("capacity:" + buf.capacity());
    }

The output result shows the range change of capacity expansion

capacity:10
capacity:64
capacity:128
View the source code to AbstractByteBuf One of the most important subclasses, ByteBuf The common properties and functions of are implemented in this.

ByteBuf buf = Unpooled.buffer(10);
System.out.println("capacity: " + buf.capacity());
for (int i = 0; i < 11; i++) {
    buf.writeByte(i);
}
----------------------------------------------------   

[ByteBuf class]
public abstract ByteBuf writeByte(int value);

Press and hold Ctrl+Alt Shortcut key

[AbstractByteBuf Subclass]
---------------------------------------------------- 
    @Override
    public ByteBuf writeByte(int value) {
        // Make sure there is enough writable space
        ensureWritable0(1);
        // Write data
        _setByte(writerIndex++, value);
        return this;
    }
    
    // The parameter is the minimum write data size
    final void ensureWritable0(int minWritableBytes) {
        final int writerIndex = writerIndex();
        // Target capacity = current write index + minimum write data size
        final int targetCapacity = writerIndex + minWritableBytes;
        // Sufficient capacity without expansion
        if (targetCapacity <= capacity()) {
            ensureAccessible();
            return;
        }
        // When the capacity is insufficient, an exception is thrown if the target capacity exceeds the maximum capacity
        if (checkBounds && targetCapacity > maxCapacity) {
            ensureAccessible();
            throw new IndexOutOfBoundsException(String.format(
                    "writerIndex(%d) + minWritableBytes(%d) exceeds maxCapacity(%d): %s",
                    writerIndex, minWritableBytes, maxCapacity, this));
        }
		
		// Expansion logic
        // Gets the size of the writable space
        final int fastWritable = maxFastWritableBytes();
        // If writable space > = required space, new capacity = write operation index + writable space size 
        // If the writable space < the required space, calculate the new capacity to be expanded. calculateNewCapacity method
        int newCapacity = fastWritable >= minWritableBytes ? writerIndex + fastWritable
                : alloc().calculateNewCapacity(targetCapacity, maxCapacity);

        // Adjust to the new capacity.
        
        // Generate a new ByteBuffer after calculation
        capacity(newCapacity);
    }
    
    // Gets the size of the writable space
    public int maxFastWritableBytes() {
        return writableBytes();
    }
    

[AbstractByteBufAllocator Subclass]
---------------------------------------------------- 
    // Calculate the new capacity to be expanded
    @Override
    public int calculateNewCapacity(int minNewCapacity, int maxCapacity) {
        // Verify parameter validity
        checkPositiveOrZero(minNewCapacity, "minNewCapacity");
        if (minNewCapacity > maxCapacity) {
            throw new IllegalArgumentException(String.format(
                    "minNewCapacity: %d (expected: not greater than maxCapacity(%d)",
                    minNewCapacity, maxCapacity));
        }
        // The dividing point of expansion mode is 4M
        final int threshold = CALCULATE_THRESHOLD; // 4 MiB page

        if (minNewCapacity == threshold) {
            return threshold;
        }

        // If over threshold, do not double but just increase by threshold.,
        // If the required capacity is greater than 4M, the capacity shall be expanded step by step 
        //   Example: minNewCapacity = 5M  
        if (minNewCapacity > threshold) {
            // newCapacity = 5 / 4 * 4 = 4M ensure it is a multiple of 4
            int newCapacity = minNewCapacity / threshold * threshold;
            if (newCapacity > maxCapacity - threshold) {
                newCapacity = maxCapacity;
            } else {
                // newCapacity = 4 + 4 = 8M;
                newCapacity += threshold;
            }
            return newCapacity;
        }

        // Not over threshold. Double up to 4 MiB, starting from 64.
        // If the required capacity is greater than 4M, expand the capacity according to the multiple of 64, and find the multiple closest to 64 of the required capacity
        int newCapacity = 64;
        while (newCapacity < minNewCapacity) {
            newCapacity <<= 1;
        }
        
        // Guaranteed within the maximum acceptable capacity
        return Math.min(newCapacity, maxCapacity);
    }

4) Advantage

A pool method to improve memory utilization

B proposes the integration scheme of composite buffer

C adds an index to separate reading and writing, which is more convenient to use

D solves the problem of fixed ByteBuffer length and adds a capacity expansion mechanism

E recycles objects by reference counting

Posted on Wed, 24 Nov 2021 03:08:26 -0500 by saqibb