Emphasis: This article is a reprint of the article, and the link is: javadoop , this big guy's articles are of great value. Click the original text to see the experience more
register operation of Channel
After the foreshadowing in front, we already have a certain foundation. Let's start to rub together the contents we learned in front. In this section, we will introduce the register operation. This step is actually very key and is very important for our source code analysis.
register
Starting from the connect() method in EchoClient or the bind(port) method in EchoServer, we will go to the initAndRegister() method:
final ChannelFuture initAndRegister() {
Channel channel = null;
try {
// 1
channel = channelFactory.newChannel();
// 2 there are some differences between Bootstrap and ServerBootstrap
init(channel);
} catch (Throwable t) {
...
}
// What we want to say here is this line
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
return regFuture;
}
We have been in touch with the initAndRegister() method twice. We introduced 1 earlier ️⃣ Channel instantiation. During the instantiation process, the instantiation of Unsafe and pipeline inside the channel will be performed, as shown in 2 above ️⃣ In the init(channel) method, a handler will be added to the pipeline (the pipeline is head + channelizer + tail at this time).
In this section, we will finally uncover the initChannel method in ChannelInitializer~~~
Now, let's move on and look at 3 ️⃣ register step:
ChannelFuture regFuture = config().group().register(channel);
As we said, register is a key step. It occurs after the channel is instantiated. Let's recall some situations in the current channel:
Instantiate the Channel at the bottom of JDK, set non blocking, instantiate Unsafe, instantiate pipeline, and add head, tail and a ChannelInitializer instance to the pipeline.
The above config().group() method returns an instance of the NioEventLoopGroup that is instantiated before, and then calls its register(channel) method:
// MultithreadEventLoopGroup
@Override
public ChannelFuture register(Channel channel) {
return next().register(channel);
}
The next() method is very simple, that is, select a thread in the thread pool (remember chooserFactory), that is, select a NioEventLoop instance. At this time, we will enter the NioEventLoop.
The register(channel) method of NioEventLoop is implemented in its parent class SingleThreadEventLoop:
@Override
public ChannelFuture register(Channel channel) {
return register(new DefaultChannelPromise(channel, this));
}
The above code instantiates a Promise and brings in the current channel:
@Override
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
// promise is associated with channel. Channel holds an Unsafe instance, and the register operation is encapsulated in Unsafe
promise.channel().unsafe().register(this, promise);
return promise;
}
Get the Unsafe instance associated with channel and call its register method:
As we said, Unsafe is specifically used to encapsulate the underlying implementation. Of course, there is no such "underlying" here
// AbstractChannel#AbstractUnsafe
@Override
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
...
// Set the eventLoop instance to the channel, and the channel will have eventLoop from then on
// I think this step is actually very critical, because all subsequent asynchronous operations in the channel must be submitted to the eventLoop for execution
AbstractChannel.this.eventLoop = eventLoop;
// If the thread initiating the register action is the thread in the eventLoop instance, register 0 (promise) is called directly
// For us, it won't enter this branch,
// The reason for this branch is that we can unregister and then register. We'll take a closer look later
if (eventLoop.inEventLoop()) {
register0(promise);
} else {
try {
// Otherwise, submit the task to eventLoop, and the thread in eventLoop will be responsible for calling register0(promise)
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
...
}
}
}
Here, we need to understand that the Thread instance has not been instantiated in NioEventLoop.
These steps involve several classes: NioEventLoop, Promise, Channel, Unsafe, etc. we should carefully sort out their relationships.
For the register operation we mentioned earlier, after submitting to eventLoop, we will directly return the promise instance. The remaining register0 is an asynchronous operation, which is completed by the NioEventLoop instance.
Let's not continue to analyze the register0(promise) method. First introduce the threads in the NioEventLoop, and then come back to introduce the register0 method.
Once the Channel instance register s a NioEventLoop instance in the NioEventLoopGroup instance, all subsequent operations of the Channel are completed by the NioEventLoop instance.
This is also very simple, because the Selector instance is in the NioEventLoop instance. Once the Channel instance is registered in a Selector instance, of course, NIO events can only be processed in this instance.
NioEventLoop workflow
Earlier, when analyzing the instantiation of thread pool, we said that Java threads are not started in NioEventLoop. Here we will carefully analyze the method called eventLoop.execute(runnable) invoked in the register process, which is in the parent class SingleThreadEventExecutor:
@Override
public void execute(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
// Judge whether the thread adding the task is the thread in the current EventLoop
boolean inEventLoop = inEventLoop();
// Add tasks to the taskQueue described earlier,
// If the taskQueue is full (the default size is 16), as we said before, the default policy is to throw an exception
addTask(task);
if (!inEventLoop) {
// If the task is not submitted by the NioEventLoop internal thread, judge whether the next thread has been started. If not, start the thread
startThread();
if (isShutdown() && removeTask(task)) {
reject();
}
}
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}
The original method to start the thread in NioEventLoop is here.
In addition, the register operation mentioned in the previous section goes into the taskQueue, so it is actually classified into the category of non IO operations.
The following is the source code of startThread. Judge whether the thread has started to decide whether to start:
private void startThread() {
if (state == ST_NOT_STARTED) {
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
try {
doStartThread();
} catch (Throwable cause) {
STATE_UPDATER.set(this, ST_NOT_STARTED);
PlatformDependent.throwException(cause);
}
}
}
}
Let's take a look at the doStartThread() method according to the previous idea and according to the situation that the thread is not started:
private void doStartThread() {
assert thread == null;
// Is the executor here familiar to you? It is the instance of threadpertask executor passed in when we instantiated NioEventLoop at the beginning. It is the kind of executor that creates a thread every time a task comes.
// Once we call its execute method, it will create a new Thread, so the Thread instance will finally be created here
executor.execute(new Runnable() {
@Override
public void run() {
// Here, set the thread created in "executor" as the thread of NioEventLoop!!!
thread = Thread.currentThread();
if (interrupted) {
thread.interrupt();
}
boolean success = false;
updateLastExecutionTime();
try {
// Execute the run() method of SingleThreadEventExecutor, which is implemented in NioEventLoop
SingleThreadEventExecutor.this.run();
success = true;
} catch (Throwable t) {
logger.warn("Unexpected exception from an event executor: ", t);
} finally {
// ... we just ignore the code here
}
}
});
}
After the above thread is started, it will execute the run() method in NioEventLoop, which is a very important method. This method is certainly not so easy to end. It must be like the Worker of JDK thread pool to continuously cycle to obtain new tasks. It needs to constantly do the select operation and poll the taskQueue.
Let's take a brief look at its source code. We won't introduce it in depth here:
@Override
protected void run() {
// The code is nested in a for loop
for (;;) {
try {
// selectStrategy is finally coming in handy
// It has two values, one is CONTINUE and the other is SELECT
// For this code, let's analyze it.
// 1. If taskQueue is not empty, that is, hasTasks() returns true,
// Then execute selectNow() once, and the method will not block
// 2. If hasTasks() returns false, execute the SelectStrategy.SELECT branch,
// select(...), which is blocked
// It is well understood that whether blocking can be performed is determined according to whether there are tasks in the queue
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.SELECT:
// If! hasTasks(), then go to the select branch, where the select is blocked
select(wakenUp.getAndSet(false));
if (wakenUp.get()) {
selector.wakeup();
}
default:
}
cancelledKeys = 0;
needsToSelectAgain = false;
// By default, the value of ioRatio is 50
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
// If ioRatio is set to 100, the IO operation is performed first, and then the task in taskQueue is executed in the finally block
try {
// 1. Perform IO operation. After the previous select, some channel s may need to be processed.
processSelectedKeys();
} finally {
// 2. Execute non IO tasks, that is, tasks in taskQueue
runAllTasks();
}
} else {
// If ioRatio is not 100, the time-consuming of non IO operations is limited according to the time-consuming of IO operations
final long ioStartTime = System.nanoTime();
try {
// Perform IO operation
processSelectedKeys();
} finally {
// According to the time consumed by IO operations, calculate how much time can be spent executing non IO operations (RUNALL tasks)
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
The above code is the core of NioEventLoop. Here are two points:
- First, we will decide whether to execute selectNow() or select(oldWakenUp) according to the results of hasTasks(), which should be easy to understand. If a task is waiting, you should use non blocking selectNow(). If no task is waiting, you can use the select operation with blocking.
- ioRatio controls the time proportion of IO operations:
- If it is set to 100%, the IO operation is performed first, and then the tasks in the task queue are executed.
- If it is not 100%, perform IO operations first, and then perform tasks in taskQueue, but you need to control the total time of executing tasks. That is, the time that non IO operations can occupy is calculated from ioRatio and the time-consuming of this IO operation.
Let's not care about the details of the select(oldWakenUp), processSelectedKeys() and runAllTasks(...) methods, but just understand what they do respectively.
Come back, we submitted the register task to NioEventLoop when registering. This is the first task received by NioEventLoop, so here we will instantiate the Thread and start it, and then enter the run method in NioEventLoop.
Continue register
Let's go back to the previous register0(promise) method. We know that the register task enters the taskQueue of NioEventLoop, and then starts the thread in NioEventLoop. The thread will poll the taskQueue, and then execute the register task.
Note that this method is executed by the thread in eventLoop:
// AbstractChannel
private void register0(ChannelPromise promise) {
try {
...
boolean firstRegistration = neverRegistered;
// ***Perform the underlying operation of JDK: register the Channel with the Selector***
doRegister();
neverRegistered = false;
registered = true;
// Here, even registered
// This step is also critical because it involves the init(channel) of the ChannelInitializer
// As we said before, the init method will add the handlers added inside the ChannelInitializer to the pipeline
pipeline.invokeHandlerAddedIfNeeded();
// Set the current promise status to success
// Because the current register method is executed in the thread in eventLoop, the thread submitting the register operation needs to be notified
safeSetSuccess(promise);
// The current register operation has been successful. This event should be on the pipeline
// All handler s who care about register events perceive that they throw an event into the pipeline
pipeline.fireChannelRegistered();
// active here means that the channel has been opened
if (isActive()) {
// If the channel executes register for the first time, the fire ChannelActive event is triggered
if (firstRegistration) {
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
// The channel has been register ed before,
// Here, let the channel immediately monitor the op in the channel_ Read event
beginRead();
}
}
} catch (Throwable t) {
...
}
}
Let's talk about the doRegister() method above first, and then talk about pipeline.
@Override
protected void doRegister() throws Exception {
boolean selected = false;
for (;;) {
try {
// The register method of Channel in JDK is attached:
// public final SelectionKey register(Selector sel, int ops, Object att) {...}
selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
return;
} catch (CancelledKeyException e) {
...
}
}
}
We can see that the register operation at the bottom of the JDK is performed here, and the socketchannel (or ServerSocketChannel) is registered in the Selector. We can see that the listening set here is set to 0, that is, nothing is listened to.
Of course, it means that the listening set of this selectionKey must be modified somewhere in the future, otherwise nothing can be done
Let's focus on the pipeline operation. When we introduced the pipeline of NioSocketChannel, we said that our pipeline now looks like this:
Now, we will see that LoggingHandler and EchoClientHandler will be added to the pipeline.
Let's continue to look at the code. After the register succeeds, the following operations are performed:
pipeline.invokeHandlerAddedIfNeeded();
You can trace that this step will be executed to the handlerAdded method of the ChannelInitializer instance in pipeline, and its init(context) method will be executed here:
@Override
public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
if (ctx.channel().isRegistered()) {
initChannel(ctx);
}
}
Then let's take a look at initChannel(ctx). Here comes the init(channel) method we introduced earlier:
private boolean initChannel(ChannelHandlerContext ctx) throws Exception {
if (initMap.putIfAbsent(ctx, Boolean.TRUE) == null) { // Guard against re-entrance.
try {
// 1. Add our customized handlers to pipeline
initChannel((C) ctx.channel());
} catch (Throwable cause) {
...
} finally {
// 2. Delete the ChannelInitializer instance from the pipeline
remove(ctx);
}
return true;
}
return false;
}
As we said earlier, after the init(channel) of the ChannelInitializer is executed, the handlers added internally will enter the pipeline, and then the ChannelInitializer instance will be deleted from the pipeline in the final block above. At this time, the pipeline will be established, as shown in the following figure:
In fact, there is another problem here. What if we add a ChannelInitializer instance to ChannelInitializer? You can consider this situation.
After the pipeline is established, we will continue to run to this sentence:
pipeline.fireChannelRegistered();
As long as we know the fireChannelRegistered() method, we'll know what's going on when we encounter other methods such as fireChannelActive() and fireXxx(), which are similar. Let's see what happens to this Code:
// DefaultChannelPipeline
@Override
public final ChannelPipeline fireChannelRegistered() {
// Note that the reference here is head
AbstractChannelHandlerContext.invokeChannelRegistered(head);
return this;
}
In other words, we throw a channelRegistered event into the pipeline. The register here belongs to the Inbound event. The next thing the pipeline needs to do is to execute the channelRegistered() method in the handlers of the Inbound type in the pipeline.
From the above code, we can see that after the channelRegistered event is thrown into the pipeline, the first handler processed is head.
Next, let's follow the code. At this time, we come to the processing of the first node head of pipeline:
// AbstractChannelHandlerContext
// next is head at this time
static void invokeChannelRegistered(final AbstractChannelHandlerContext next) {
EventExecutor executor = next.executor();
// Execute invokeChannelRegistered() of head
if (executor.inEventLoop()) {
next.invokeChannelRegistered();
} else {
executor.execute(new Runnable() {
@Override
public void run() {
next.invokeChannelRegistered();
}
});
}
}
In other words, the head.invokeChannelRegistered() method will be executed first, and it will be executed in the taskQueue in NioEventLoop:
// AbstractChannelHandlerContext
private void invokeChannelRegistered() {
if (invokeHandler()) {
try {
// The handler() method returns head
((ChannelInboundHandler) handler()).channelRegistered(this);
} catch (Throwable t) {
notifyHandlerException(t);
}
} else {
fireChannelRegistered();
}
}
Let's look at the channelRegistered method of head:
// HeadContext
@Override
public void channelRegistered(ChannelHandlerContext ctx) throws Exception {
// 1. This step is the head's handling of the channelRegistered event. There's nothing we should care about
invokeHandlerAddedIfNeeded();
// 2. Propagate Inbound events backward
ctx.fireChannelRegistered();
}
Then head will execute the fireChannelRegister() method:
// AbstractChannelHandlerContext
@Override
public ChannelHandlerContext fireChannelRegistered() {
// It's crucial here
// The findContextInbound() method will find the next handler of Inbound type along the pipeline
invokeChannelRegistered(findContextInbound());
return this;
}
Note: pipeline.fireChannelRegistered() throws the channelRegistered event into the pipeline, and the handlers in the pipeline are ready to handle the event. context.fireChannelRegistered() is a handler that propagates back to the next handler after processing.
The names of their two methods are the same, but they come from different classes.
findContextInbound() will find the handler of the next Inbound type, and then repeat the above methods.
I don't think the above code needs to be too tangled. In short, start from the head, find all inbound handlers in turn, and execute their channelRegistered(ctx) operation.
Having said so much, our register operation is really completed.
Next, let's go back to the initAndRegister method:
final ChannelFuture initAndRegister() {
Channel channel = null;
try {
channel = channelFactory.newChannel();
init(channel);
} catch (Throwable t) {
...
}
// We finished the line above
ChannelFuture regFuture = config().group().register(channel);
// If an error occurs during register
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
// The source code makes it clear that if you go here, you can connect() or bind() later because of two situations:
// 1. If the register action is initiated in eventLoop, the register must have been completed by the time you get here
// 2. If the register task has been submitted to the eventLoop, that is, to the taskQueue in the eventLoop,
// Since the subsequent connect or bind will also enter the queue of the same eventLoop, the connect or bind will not be executed until the register succeeds
return regFuture;
}
We should know that no matter the NioServerSocketChannel on the server side or the NioSocketChannel on the client side, the initAndRegister method will be entered first when bind ing or connect ing, so what we said above is common to both.
You should remember that the register operation is very important. You should know what has been done in this step. After the register operation, it will enter the bind or connect operation.
Analysis of connect process and bind process
The register operation described above is very critical. It has established a lot of things. It is the starting point for NioSocketChannel and NioServerSocketChannel in Netty.
In this section, we will talk about the connect operation and bind operation after register. This section is very simple.
connect process analysis
For the client NioSocketChannel, after the previous register is completed, it will start to connect. This step will connect to the server.
private ChannelFuture doResolveAndConnect(final SocketAddress remoteAddress, final SocketAddress localAddress) {
// The register operation is completed here
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
// Here, we don't worry about whether the register operation is isDone()
if (regFuture.isDone()) {
if (!regFuture.isSuccess()) {
return regFuture;
}
// Look here
return doResolveAndConnect0(channel, remoteAddress, localAddress, channel.newPromise());
} else {
....
}
}
Here, everyone points in all the way, so I won't waste space. Finally, we will come to the connect method of AbstractChannel:
@Override
public ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
return pipeline.connect(remoteAddress, promise);
}
We can see that the connect operation is performed by pipeline. After entering the pipeline, we will find that the Outbound operation of connect starts from the tail of the pipeline:
The register operation we introduced earlier is Inbound, starting from the head
@Override
public final ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
return tail.connect(remoteAddress, promise);
}
The next step is the pipeline operation. Starting from tail, execute the connect(...) method of Outbound handlers on the pipeline. Where does the real underlying connect operation occur? Remember the picture of our pipeline?
Start from tail and look for out type handlers. Each time you pass through a handler, execute the connect() method inside, and finally go to the head. Because the head is also of Outbound type, the connect operation we need is in the head, which will be responsible for calling the connect method provided in unsafe:
// HeadContext
public void connect(
ChannelHandlerContext ctx,
SocketAddress remoteAddress, SocketAddress localAddress,
ChannelPromise promise) throws Exception {
unsafe.connect(remoteAddress, localAddress, promise);
}
Next, let's take a look at the so-called underlying operations of connect in the unsafe class:
// AbstractNioChannel.AbstractNioUnsafe
@Override
public final void connect(
final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise promise) {
......
boolean wasActive = isActive();
// Click inside to see the doConnect method
// In this step, you will do SocketChannel connect at the bottom of JDK, and then set interestOps to SelectionKey.OP_CONNECT
// The return value represents whether the connection has been successful
if (doConnect(remoteAddress, localAddress)) {
// Handle successful connections
fulfillConnectPromise(promise, wasActive);
} else {
connectPromise = promise;
requestedRemoteAddress = remoteAddress;
// The following code is very simple when dealing with connection timeout
// The timing task function of NioEventLoop is used here, which we haven't introduced before, because I don't think it's very important
int connectTimeoutMillis = config().getConnectTimeoutMillis();
if (connectTimeoutMillis > 0) {
connectTimeoutFuture = eventLoop().schedule(new Runnable() {
@Override
public void run() {
ChannelPromise connectPromise = AbstractNioChannel.this.connectPromise;
ConnectTimeoutException cause =
new ConnectTimeoutException("connection timed out: " + remoteAddress);
if (connectPromise != null && connectPromise.tryFailure(cause)) {
close(voidPromise());
}
}
}, connectTimeoutMillis, TimeUnit.MILLISECONDS);
}
promise.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isCancelled()) {
if (connectTimeoutFuture != null) {
connectTimeoutFuture.cancel(false);
}
connectPromise = null;
close(voidPromise());
}
}
});
}
} catch (Throwable t) {
promise.tryFailure(annotateConnectException(t, remoteAddress));
closeIfClosed();
}
}
If the doConnect method above returns false, what happens next?
In the register operation described in the previous section, the channel has registered to the selector, but the interestOps is set to 0, that is, nothing is monitored.
In the doConnect method above, we can see that after calling the underlying connect method, it will set interestOps to SelectionKey.OP_CONNECT.
The rest is about NioEventLoop. Remember the run() method of NioEventLoop? In other words, after the connection here is successful, the TCP connection will be established, and subsequent operations will be handled by the processSelectedKeys() method in the NioEventLoop.run() method.
bind process analysis
After completing the connect process, let's briefly look at the bind process:
private ChannelFuture doBind(final SocketAddress localAddress) {
// **initAndRegister mentioned earlier**
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
if (regFuture.cause() != null) {
return regFuture;
}
if (regFuture.isDone()) {
// If the register action has been completed, perform the bind operation
ChannelPromise promise = channel.newPromise();
doBind0(regFuture, channel, localAddress, promise);
return promise;
} else {
......
}
}
Then keep looking inside and you will see that the bind operation is also completed by pipeline:
// AbstractChannel
@Override
public ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) {
return pipeline.bind(localAddress, promise);
}
Like connect, the bind operation is of Outbound type, so it starts with tail:
@Override
public final ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) {
return tail.bind(localAddress, promise);
}
The last bind operation goes to the head. The head calls the bind method provided by unsafe:
@Override
public void bind(
ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise)
throws Exception {
unsafe.bind(localAddress, promise);
}
Interested readers take a look at the bind method in unsafe. It is very simple. The bind operation is not an asynchronous method, so let's introduce it here.
This section is very simple. I want to introduce you to the routines of various operations in Netty.