Source code analysis of start and stop of Spring Boot Dubbo application

Author: Zhang Huxing source: Dubbo official blog

Background

Dubbo Spring Boot works to simplify Dubbo RPC Development of the framework in the Spring Boot application scenario. It also integrates the Spring Boot feature:

  • Automatic assembly (e.g. annotation driven, automatic assembly, etc.)

  • Production ready (e.g. safety, health check, external configuration, etc.)

DubboConsumer start analysis

Have you ever thought about a question? The DubboConsumerDemo application in the incregator Dubbo spring boot project is one line of code. After the main method is executed, why doesn't it exit directly?

@SpringBootApplication(scanBasePackages = "com.alibaba.boot.dubbo.demo.consumer.controller")
public class DubboConsumerDemo {

   public static void main(String[] args) {
       SpringApplication.run(DubboConsumerDemo.class,args);
   }

}

In fact, to answer such a question, we first need to abstract this question, that is, a JVM process, under what circumstances will it exit?

Taking Java 8 as an example, by referring to the JVM language specification [1], it is clearly described in Chapter 12.8:

A program terminates all its activity and exits when one of two things happens:

  • All the threads that are not daemon threads terminate.

  • Some thread invokes the exit method of class Runtime or class System, and the exitoperation is not forbidden by the security manager.

In other words, there are only two situations that lead to the exit of the JVM:

  1. All non daemon Process completely terminated

  2. A thread called System.exit() or Runtime.exit()

Therefore, in view of the above situation, we judge that there must be a non daemon thread that does not exit. We know that we can see all thread information through jstack, including whether they are daemon threads. We can find out those threads that are not deamon threads through jstack.

jstack 57785 | grep tid | grep -v "daemon"
"container-0" #37 prio=5 os_prio=31 tid=0x00007fbe312f5800 nid=0x7103 waiting on condition  [0x0000700010144000]
"container-1" #49 prio=5 os_prio=31 tid=0x00007fbe3117f800 nid=0x7b03 waiting on condition  [0x0000700010859000]
"DestroyJavaVM" #83 prio=5 os_prio=31 tid=0x00007fbe30011000 nid=0x2703 waiting on condition  [0x0000000000000000]
"VM Thread" os_prio=31 tid=0x00007fbe3005e800 nid=0x3703 runnable
"GC Thread#0" os_prio=31 tid=0x00007fbe30013800 nid=0x5403 runnable
"GC Thread#1" os_prio=31 tid=0x00007fbe30021000 nid=0x5303 runnable
"GC Thread#2" os_prio=31 tid=0x00007fbe30021800 nid=0x2d03 runnable
"GC Thread#3" os_prio=31 tid=0x00007fbe30022000 nid=0x2f03 runnable
"G1 Main Marker" os_prio=31 tid=0x00007fbe30040800 nid=0x5203 runnable
"G1 Conc#0" os_prio=31 tid=0x00007fbe30041000 nid=0x4f03 runnable
"G1 Refine#0" os_prio=31 tid=0x00007fbe31044800 nid=0x4e03 runnable
"G1 Refine#1" os_prio=31 tid=0x00007fbe31045800 nid=0x4d03 runnable
"G1 Refine#2" os_prio=31 tid=0x00007fbe31046000 nid=0x4c03 runnable
"G1 Refine#3" os_prio=31 tid=0x00007fbe31047000 nid=0x4b03 runnable
"G1 Young RemSet Sampling" os_prio=31 tid=0x00007fbe31047800 nid=0x3603 runnable
"VM Periodic Task Thread" os_prio=31 tid=0x00007fbe31129000 nid=0x6703 waiting on condition

Here, all thread summaries are found by grep tid, and rows without the daemon keyword are found by grep-v

Through the above results, we found some information:

  • There are two threads, container-0 and container-1, which are very suspicious. They are non daemon threads and in the wait state

  • There are some GC related threads, VM led threads, and non daemon threads, but they are most likely the JVM's own threads, which are ignored here for the time being.

In conclusion, we can infer that the JVM did not exit because of container-0 and container-1. Now let's search the source code to find out who created these two threads.

Through the source code analysis of spring boot, we found the following code in startdaemoniwatthread of org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer

private void startDaemonAwaitThread() {
       Thread awaitThread = new Thread("container-" + (containerCounter.get())) {

           @Override
           public void run() {
               TomcatEmbeddedServletContainer.this.tomcat.getServer().await();
           }
       };
       awaitThread.setContextClassLoader(getClass().getClassLoader());
       awaitThread.setDaemon(false);
       awaitThread.start();

}

Add a breakpoint in this method. See call stack:

initialize:115, TomcatEmbeddedServletContainer (org.springframework.boot.context.embedded.tomcat)
<init>:84, TomcatEmbeddedServletContainer (org.springframework.boot.context.embedded.tomcat)
getTomcatEmbeddedServletContainer:554, TomcatEmbeddedServletContainerFactory (org.springframework.boot.context.embedded.tomcat)
getEmbeddedServletContainer:179, TomcatEmbeddedServletContainerFactory (org.springframework.boot.context.embedded.tomcat)
createEmbeddedServletContainer:164, EmbeddedWebApplicationContext (org.springframework.boot.context.embedded)
onRefresh:134, EmbeddedWebApplicationContext (org.springframework.boot.context.embedded)
refresh:537, AbstractApplicationContext (org.springframework.context.support)
refresh:122, EmbeddedWebApplicationContext (org.springframework.boot.context.embedded)
refresh:693, SpringApplication (org.springframework.boot)
refreshContext:360, SpringApplication (org.springframework.boot)
run:303, SpringApplication (org.springframework.boot)
run:1118, SpringApplication (org.springframework.boot)
run:1107, SpringApplication (org.springframework.boot)
main:35, DubboConsumerDemo (com.alibaba.boot.dubbo.demo.consumer.bootstrap)

As you can see, in the process of starting spring boot application, Tomcat exposes HTTP service by default, so the above methods are executed. All threads started by Tomcat are the daemon threads by default, such as Acceptor listening to requests, worker thread pool, etc. if there is no control here, the JVM will exit after the start. Therefore, it is necessary to start a thread explicitly and wait continuously under certain conditions to avoid thread exit. Source code analysis of the whole process of Spring Boot 2.x startup (all) , this article is recommended for you to read.

Let's dig deeper into how the thread does not exit in Tomcat's this.tomcat.getServer().await(). For the convenience of reading, irrelevant code is removed here.

public void await() {
    // ...
    if( port==-1 ) {
        try {
            awaitThread = Thread.currentThread();
            while(!stopAwait) {
                try {
                    Thread.sleep( 10000 );
                } catch( InterruptedException ex ) {
                    // continue and check the flag
                }
            }
        } finally {
            awaitThread = null;
        }
        return;
    }
    // ...
}

In the await method, the current thread actually checks the stopAwait variable every 10 seconds in a while loop. It is a To ensure that the current thread can see the change immediately after it is modified by another thread. If there is no change, it will stay in the while loop. This is why the thread does not exit, that is, the whole spring boot application does not exit.

Because the spring boot application starts two ports, 8080 and 8081(management port), and actually starts two Tomcat, there will be two threads, container-0 and container-1.

Next, let's see how the spring boot application exits?

DubboConsumer exit analysis

As mentioned in the previous description, there is a thread continuously checking the stopAwait variable, so we naturally think that when we Stop, there should be a thread to modify the stopAwait and break the while loop, so who is modifying the variable?

Through the source code analysis, we can see that only one method has modified stopAwait, that is, org.apache.catalina.core.standardserver ා stopAwait. Let's add a breakpoint here to see who is calling.

Note that when we add a breakpoint in the Debug mode of Intellij IDEA, we need to use kill-s INT $PID or kill-s TERM $PID on the command line to trigger the breakpoint. Clicking the Stop button on the IDE will not trigger the breakpoint. This is a bug in IDEA. Debugging Bug in IDEA is really amazing! Let's take a look at this recommendation.

You can see that a thread named Thread-3 called this method:

stopAwait:390, StandardServer (org.apache.catalina.core)
stopInternal:819, StandardServer (org.apache.catalina.core)
stop:226, LifecycleBase (org.apache.catalina.util)
stop:377, Tomcat (org.apache.catalina.startup)
stopTomcat:241, TomcatEmbeddedServletContainer (org.springframework.boot.context.embedded.tomcat)
stop:295, TomcatEmbeddedServletContainer (org.springframework.boot.context.embedded.tomcat)
stopAndReleaseEmbeddedServletContainer:306, EmbeddedWebApplicationContext (org.springframework.boot.context.embedded)
onClose:155, EmbeddedWebApplicationContext (org.springframework.boot.context.embedded)
doClose:1014, AbstractApplicationContext (org.springframework.context.support)
run:929, AbstractApplicationContext$2 (org.springframework.context.support)

Through the source code analysis, it was originally executed through the ShutdownHook registered by Spring

@Override
public void registerShutdownHook() {
   if (this.shutdownHook == null) {
       // No shutdown hook registered yet.
       this.shutdownHook = new Thread() {
           @Override
           public void run() {
               synchronized (startupShutdownMonitor) {
                   doClose();
               }
           }
       };
       Runtime.getRuntime().addShutdownHook(this.shutdownHook);
   }

}

By consulting the Java API document [2], we can know that the shutdown hook will execute in the following two cases

The Java virtual machine shuts down in response to two kinds of events:

  • The program exits normally, when the last non-daemon thread exits or when the exit(equivalently, System.exit) method is invoked, or

  • The virtual machine is terminated in response to a user interrupt, such as typing ^C, or a system-wide event, such as user logoff or system shutdown.

  1. System.exit() method called

  2. Respond to external signals, such as Ctrl+C (SIGINT signal is actually sent), or SIGTERM signal (SIGTERM signal is sent by kill $PID by default)

Therefore, the normal application will execute the above shutdown hook in the process of stopping (except kill-9$PID). Its function is not only to shut down tomcat, but also to carry out other cleaning work, which will not be described here.

summary

  1. In the process of DubboConsumer startup, check the status of variables by starting an independent non daemon thread loop to ensure that the process does not exit

  2. In the process of DubboConsumer stopping, the state of variables is modified by executing the shutdown hook of spring container, so that the program exits normally

problem

In the example of DubboProvider, we can see that the Provider does not start Tomcat to provide HTTP services, so how can we achieve non exit? We will answer this question in the next article.

Egg

Run the following unit test in IntelliJ idea to create a thread to perform the operation of sleeping for 1000 seconds. We are surprised to find that the code exits before the thread finishes executing. Why? (the created thread is a non daemon thread)

@Test
public void test() {
   new Thread(new Runnable() {
       @Override
       public void run() {
           try {
               Thread.sleep(1000000);
           } catch (InterruptedException e) {
               e.printStackTrace();
           }
       }
   }).start();
}

[1] https://docs.oracle.com/javase/specs/jls/se8/html/jls-12.html#jls-12.8

[2] https://docs.oracle.com/javase/8/docs/api/java/lang/Runtime.html#addShutdownHook

Focus on the Java technology stack WeChat official account, reply the key word in the background: Dubbo, you can get more Dubbo dry cargo with long stack.

Recommend to my blog to read more:

1.Java JVM, collection, multithreading, new features series

2.Spring MVC, Spring Boot, Spring Cloud series tutorials

3.Maven, Git, Eclipse, Intellij IDEA series tools tutorial

4.Latest interview questions of Java, backend, architecture, Alibaba and other large factories

Feel good, don't forget to like + forward!

Tags: Java Tomcat Spring Dubbo jvm

Posted on Tue, 05 May 2020 15:16:18 -0400 by PHPFEEDER