Netty-Messaging Working Mechanism

After a business invokes write, it is processed by the ChannelPipeline responsibility chain, and the message is delivered to the message sending buffer to be sent. After calling flush, the real send operation is performed. The underlying layer sends the message to the network by calling the SocketChannel of Java NIO for a non-blocking write operation.

Principle and Source Code Analysis of WriteAndFlushTask
ChannelOutboundBuffer Principle and Source Code Analysis
* Message Sending Source Analysis
High and low water level control for message sending

WriteAndFlushTask Principle and Source Code Analysis

To maximize performance, Netty uses a serial unlocked design that serializes operations inside I/O threads to avoid performance degradation due to multithreaded competition.On the surface, the CPU utilization of the serialized design seems to be low and the concurrency level is insufficient.However, by adjusting the thread parameters of the NIO thread pool, multiple serialized threads can be started running in parallel at the same time, which is a partial serial thread design with more performance concerns than the "one queue, multiple worker threads" model. Portal: Netty-What is Serial Unlocking
When a user initiates a write operation, Netty decides that if it is not a NioEventLoop(I/O thread), it encapsulates the sending message as a WriteTask, and the task queue put into NIoEventLoop is executed by the NioEventLoop thread with the following code (AbstractChannellerHandContext class):

private void write(Object msg, boolean flush, ChannelPromise promise){
	AbstractChannelHandlerContext next = findContextOutbound();
	final Object m = pipeline.touch(msg, next);
	EventExecutor executor = next.executor;
	if(executor.inEventLoop()){
		if(flush){
			next.invokeWriteAndFlush(m, promise);
		}else{
			next.invokeWrite(m, promise);
		}
	}else{
		AbstractWriteTask task;
		if(flush){
			task = WriteAndFlushTask.newInstance(next, m, promise);
		}else{
			task = WriteTask.newInstance(next, m, promise);
		}
		safeExecute(executor, task, promise, m);
	}
}

Netty's NoEventLoop thread maintains a Queue taskQueue internally, which handles both network I/O reads and writes as well as Tasks related to network reads and writes (including user-defined Tasks) coded as follows (SingleThreadEventExecutor class):

public void execute(Runnable task){
	if(task == null){
		throw new NullPointerException("task");
	}
	boolean inEventLoop = inEventLoop();
	addTask(task);
	if(!inEventLoop){	//Not the current thread, task queue related operations
		startThread();
		if(isShutDown() && removeTask(task)){
			reject()
		}
	}
	if(!addTaskWakesUp && wakesUpForTask(task)){
		wakeup(inEventLoop);
	}
}

NioEventLoop traverses taskQueue to perform message sending tasks with the following code (AbstractWriteTask class):

public final void run(){
	try{
		if(ESTIMATE_TASK_SIZE_ON_SUBMIT){
			ctx.pipeline.decrementPendingOutboundBytes(size);
		}
		write(ctx, msg, promise);
	}finally{

	}
}

After a series of system processing operations, the addMessage method of ChannelOutboundBuffer is finally invoked to join the sent message to the send queue.
From the above analysis, you can see that:

  • It is thread-safe for multiple business threads to concurrently invoke write-related methods, and Netty encapsulates the sending message as a Task and executes it asynchronously by I/O threads.
  • Since a single Channel is executed by its corresponding NioEventLoop thread, a WriteTask backlog can occur if the write operation calling a Channel in parallel exceeds the execution capacity of the NioEventLoop thread.
  • NioEventLoop threads need to handle network read and write operations, as well as various Tasks registered on NioEventLoop, which interact. Heavy network read and write tasks, or too many Tasks registered, can cause the other party to delay execution and cause performance problems.

ChannelOutboundBuffer Principle and Source Code Analysis

ChannelOutboundBuffer is Netty's send buffer queue that manages messages to be sent based on a list of chains, defined as follows (ChannelOutboundBuffer class):

// The first element refreshed in the cache chain table
private Entry flushedEntry;
// The first element in the cache chain table that has not been refreshed
private Entry unFlushedEntry;
// Cache tail elements in a chain table
private Entry tailEntry;
// Number of refreshes that have not yet been written to the socket
private int flushed;

static final Entry {
	private final Handle<Entry> handle;
	Entry next;
	Object msg;
	ByteBuffer[] bufs;
	ByteBuffer[] buf;
	ChannelPromise promise;
}

When a message is sent, the addMessage method of ChannelOutboundBuffer is invoked, the chain table pointer is modified, the newly added message is placed at the end, and the next pointer of the last tail message is updated to point to the newly added message with the following code:

private void addMessage(Object msg, int size, ChannelPromise promise){
	//1. Create a new entry
	Entry entry = Entry.newInstance(msg, size, total(msg), promise);
	if(tailEntry == null){	
		//2. Determine if tailEntry is null, and if NULL indicates that the list of chains is empty, set flushedEntry to null
		flushedEntry = null;
	}else{
		//3. If tailEntry is not empty, add the newly added Entry after tailEntry
		Entry tail = tailEntry;
		tail.next = entry;
	}
	//4. Set the newly added Entry to chain list tailEntry
	tailEntry = entry;
	if(unflushedEntry == null){
		//5. If unflushedEntry is empty, the element has not been refreshed.The newly added Entry must have not been refreshed.
		//Set Entry before point tounflushedEntry(The first element in the cache chain table that has not been refreshed)
		unflushedEntry = entry;
	}
}

AdMessage successfully added to ChannelOutboundBuffer requires flush to refresh to the Socket, but this method does not do a refresh to the Socket, but instead transfers the reference to unflushedEntry to the flushEntry reference, indicating that the flushedEntry will be refreshed.Because Netty provides promise, this object can cancel, so after writing, flush needs to tell promise that it can't cancel.The code is as follows:

public void addFlush(){
	//1. Get the element entry not refreshed by unflushedEntry
	//2. If entry is null, no action is taken to indicate that there are no elements to refresh
	Entry entry = unflushedEntry;
	//3. If entry is not null, there are elements that need to be refreshed
	if(entry != null){
	//4. If flushedEntry is empty indicating that there is no task currently being refreshed, set entry as the starting point for flushedEntry refresh
		if(flushedEntry == null){
			flushedEntry = entry;
		}
	//5. Loop the Entry to set the states of these Entries to non-cancel states. If the settings fail, cancel these entry nodes and subtract the byte size of this node from TotPendingSize.
		do{
			flushed++;
			if(!entry.promise.setUncancellable()){
				int pending = entry.cancel();
				decrementPendingOutboundBytes(pending, false, true);
			}
			entry = entry.next;
		}while(entry != null);
		unflushedEntry = null;
	}
}

After calling the addFlush method, Channel calls the flush0 method for a real refresh.

When sending a message, call the current method to get the original information to be sent (flushedEntry):

public Object current(){
	Entry entry = flushedEntry;
	if(entry == null){
		return null;
	}
	return entry.msg;
}

If the message is sent successfully, the remove method of ChannelOutboundBuffer is called to delete the sent message from the list of chains and update the message to be sent with the following code:

private void removeEntry(Entry e){
	//1. If flushed 0 means that all flush data in the chain table has been sent to the socket, set flushedEntry to null
	if(--flushed == 0){
		flushedEntry = null;
		if(e == tailEntry){
		//Explain that the list of chains is empty, leave both tailEntry and unflushedEntry empty
			tailEntry = null;
			unflushedEntry = null;
		}
	}else{
	//Set flushedEntry as the next node
		flushedEntry = e.next;
	}	
}

Release the ByteBuf resource after deleting the sent message from the list of chains and return to the pool for reuse if it is a ByteBuf based on memory pool allocation: If it is in non-pool mode, empty the related resource and wait for GC recycling with the following code (ChannelOutboundBuffer):

public boolean remove(){
	if(!e.cancelled){
		ReferenceCountUtil.safeRelease(msg);
		safeSuccess(promise);
	}
}

Message Sending Source Analysis

  • Send limit
    When SocketChannel cannot write all ByteBuf/ByteBuffer to the network at once, it needs to decide whether to register SelectionKey.OP_WRITE to continue sending at the next Selector poll or to loop through it at the current location until all messages have been sent before returning.
    (ii) Frequent registration of SelectionKey.OP_WRITE and wakeup Selector can affect performance; however, if TCP's send buffer is full, TCP is in KEEP-ALIVE state, messages cannot be sent out, if the number of circular sends is not controlled, messages can be sent for a long time, and Reactor threads cannot read other messages and execute queued Task s in time.Netty takes a compromise approach. If the number of bytes sent this time is greater than 0, but the message has not been sent yet, it sends in a loop. Once the number of bytes written is found to be 0, indicating that the TCP buffer is full, it makes no sense to continue sending at this time. Register SelectionKey.OP_WRITE and exit the loop, and continue sending in the next SelectionKey polling cycle with the following code (NioSocketChannel class):
protected void doWrite(ChannelOutboundBuffer in) throws Exception{
	SocketChannel ch = javaChannel();
	//Number of times to get spin, default 16
	int writeSpinCount = config().getWriteSpinCount();
	do{
		//Message Sending Code
	} while(writeSpinCount > 0){
		//If 16 spins have not completed flush, create a task to queue for execution
		incompleteWrite(writeSpinCount < 0);
	}
}
protected final void incompleteWrite(boolean setOpWrite) {
	if(setOpWrite){
		this.setOpWrite();
	}else {
        this.clearOpWrite();
        this.eventLoop().execute(this.flushTask);
    }
}
  • Different messaging strategies
    (1) If the number of ByteBuffers for the message to be sent (ChannelOutbound Buffer) equals 1, get the ByteBuffer for the message to be sent through nioBuffers[0], and complete the message sending directly by calling the SocketChannel of JDK, as follows (doWrite method of NioSocketChannel class):
for(;;){
	ByteBuffer[] nioBuffers = in.nioBuffers(1024, maxBytesPerGatheringWrite);
	int nioBufferCnt = in.nioBufferCount();
	switch(nioBufferCnt){
		case 1: {
			ByteBuffer buffer = nioBuffers[0];
			int attemptedBytes = buffer.remaining();
			final int localWrittenBytes = ch.write(buffer);
			if(localWrittenBytes <= 0){
				incompleteWrite(true);
				return;
			}
			adjustMaxBytesPerGatheringWrite(attemptedBytes, localWrittenBytes, maxBytesPerGatheringWrite);
			in.removeBytes(localWrittenBytes);
			--writeSpinCount;
			break;
		}
	}
}

(2) If the number of ByteBuffer s for a message to be sent is greater than 1, call the batch send interface of SocketChannel and write the nioBuffers array to the TCP send buffer with the following code:

default:{
	long attemptedBytes = in.nioBufferSize();
	final long localWrittenBytes = ch.write(nioBuffers, 0, nioBufferCnt);
	if(localWrittenBytes <= 0){
		incompleteWrite(true);
		return;
	}
	adjustMaxBytesPerGatheringWrite((int)attemptedBytes, (int)localWrittenBytes, maxBytesPerGatheringWrite);
	in.removeBytes(localWrittenBytes);
	--writeSpinCount;
	break;
}

(3) If the message to be sent contains 0 JDK native ByteBuffers, call the doWrite0 method of the parent AbstractNioByteChannel to send Netty's ByteBuffer a set of TCP buffers, coded as follows (AbstractNioByteChannel class):

private int doWriteInternal(ChannelOutboundBuffer in, Object msg) throws Exception{
	if(msg instanceof ByteBuf){
		ByteBuf buf = (ByteBuf) msg;
		if(!buf.isReadable()){
			in.remove();
			return 0;
		}
		final int localFlushedAmount = doWriteBytes(buf);
		if(localFlushedAmount > 0){
			in.progress(localFlushedAmount);
			if(!buf.isReadable()){
				in.remove();
			}
			return 1;
		}
	}
}
  • Sent Message Memory Release
    If the message is sent successfully, Netty releases the memory of the sent message, and the release policy varies depending on the object being sent.
    (1) If the sending object is a ByteBuffer of the JDK, then the number of sending objects to be released is calculated based on the number of bytes sent, as follows (ChannelOutboundBuffer class):
public void removeBytes(long writtenBytes){
	while(true){
		//Pre-point flushedEntry node
		Object msg = current();
		if(!(msg instanceof ByteBuf)){
			assert writtenBytes == 0;
		}
		final ByteBuf buf = (ByteBuf) msg;
		final int readerIndex = buf.readerIndex();
		final int readableBytes = buf.writerIndex() -readerIndex;
		//Compare the number of readable bytes with the total number of bytes sent. If the number of bytes sent is greater than the number of readable bytes, the current ByteBuffer has been sent completely.
		//flushedEntry Finished
		if(readableBytes <= writtenBytes ){
			if(writtenBytes != 0){
				//Update Progress
				progress(readableBytes);
				writtenBytes -= readableBytes;
			}
			//Delete the node that flushedEntry points to and move flushedEntry backwards
			remove();
		}else{
			//The flushedEntry is not finished, just update the progress
			if(writtenBytes != 0){
				buf.readerIndex(readerIndex + (int) writtenBytes);
				progress(writtenBytes);
			}
			break;
		}
	}
	clearNioBuffers();
}

(2) If the sending object is Netty's ByteBuf, get the result of sending the message by judging the isReadable of the current ByteBuf. If the sending is complete, call ChannelOutboundBuffer's remove method to delete and release the ByteBuf as follows (AbstractNioByteChannel class doWriteInternal):

final int localFlushedAmount = doWriteBytes(buf);
	if(localFlushedAmount > 0){
		in.progress(localFlushedAmount);
		if(!buf.isReadable()){
			in.remove();
		}
	return 1;
}
  • Write Half Package
    If at one time all the messages to be sent cannot be written to the TCP buffer, the cyclic writeSpinCount has not been sent yet, or a TCP zero sliding window appears during the sending process (the number of bytes written is 0), it enters the Write-in-Half-Packet mode (to prevent the NioEventLoop thread from deadly cyclic sending when the message is slow), registers SelectionKey.OP_WRITE to the corresponding SelEctor, exit the loop, and continue with the write operation during the next Selector poll with the following code (the doWrite method of the NioSocketChannel class):
if(localWrittenBytes <= 0){
	incomplete(true);
	return;
}

The registration code for SelectionKey.OP_WRITE is as follows (the AbstractNioByteChannel class setOpWrite method):

protected final void setOpWrite(){
	final SelectionKey key = selectionKey();
	if(!key.isValid()){
		return;
	}
	final int interestOps = key.interestOps();
	if((interestOps & SelectionKey.OP_WRITE) == 0){
		key.interestOps(interestOps | SelectionKey.OP_WRITE);
	}
}

Message Sending High and Low Water Level Control

In order to control the sending speed and message backlog, Netty provides a high and low water level mechanism. When the total number of bytes backlogged in the message queue to be sent reaches a high water level, modifying the status of Channel is not writable with the following code (ChannelOutboundBuffer class):

private void incrementPendingOutboundBytes(long size, boolean invokeLater){
	if(size == 0){
		return;
	}
	long newWriteBufferSize = TOTAL_PENDING_SIZE_UPDATER.addAndGet(this, size);
	if(newWriteBufferSize > channel.config().getWriteBufferHighWaterMark()){
		setUnwrite(invokeLater);
	}
}

After modifying the Channel state, call ChannelPipeline to send a notification event, and the business can listen for the event to get the link writable state in time, coded as follows (ChannelOutboundBuffer class):

private void fireChannelWritabilityChanged(boolean invokeLater){
	final ChannelPipeline pipeline = channel.pipeline();
	if(invokeLater){
		Runnable task = fireChannelWritabilityChangedTask;
		if(task == null){
			fireChannelWritabilityChangedTask = task = new Runnable(){
				@Override
				public void run(){
					pipeline.fireChannelWritabilityChanged();
				}
			};
		}
		channel.eventLoop().execute(task);
	} else{
		pipeline.fireChannelWritabilityChaned();
	}
}

After the message is sent, judge the low water level. If the current backlog of bytes to be sent reaches or falls below the low water level, modify the Channel state to be writable and send a notification event with the following code:

private	void decreamentPendingOutboundBytes(long size, boolean invokeLater, boolean notifyWritability){
	if(size == 0){
		return;
	}
	long newWriteBufferSize = TOTAL_PENDING_SIZE_UPDATER.addAndGet(this, -size);
	if(notifyWritability && newWriteBufferSize < channel.config().getWriteBufferLowWaterMark()){
		setWritable(invokeLater);
	}
}

The high and low water level mechanism prevents the sending queue from continuing to send messages when it is at a high water level, resulting in a greater backlog.

Six original articles were published. Praise 5. Visits 235
Private letter follow

Tags: Netty network socket JDK

Posted on Sun, 12 Jan 2020 20:38:33 -0500 by mdomel