26 - how does the production environment exclude and optimize the JVM?

I believe you have a general impression on the theory and practice of the JVM through the previous several lessons. This lesson will focus on the troubleshooting and optimization of the JVM, so as to have a complete understanding of the knowledge points of the JVM, so that it can be better applied to the actual work or interview.

Our interview question in this lesson is how to check the production environment?

Typical answer

If you are directly troubleshooting a JVM in a production environment, the easiest way is to use the six very practical command-line tools provided with the JDK. They are: jps, jstat, jinfo, jmap, jhat and jstack. They are all located in the bin directory of JDK, and can be run directly using command-line tools. The directory is shown below:

Let's take a look at the specific use of these tools.

1. jps (virtual machine process status tool)

jps (JVM Process Status tool) is similar to ps command in Linux. It is used to list LVMID (Local Virtual Machine IDentifier, unique ID of local virtual machine) of running JVM, execution main class of JVM, JVM startup parameters and other information. The syntax is as follows:

jps [options] [hostid]

Common options:

  • -l: It is used to output the full name of the running main class. If it is a jar package, the path of the jar package is output;
  • -q: Used to output LVMID (Local Virtual Machine Identifier, virtual machine unique ID);
  • -m: Used to output the parameters passed to the main class main() method when the virtual machine starts;
  • -v: Used to output JVM parameters at startup.

Examples:

➜  jps -l
68848
40085 org.jetbrains.jps.cmdline.Launcher
40086 com.example.optimize.NativeOptimize
40109 jdk.jcmd/sun.tools.jps.Jps
68879 org.jetbrains.idea.maven.server.RemoteMavenServer36
➜  jps -q
40368
68848
40085
40086
68879
➜  jps -m
40400 Jps -m
68848
40085 Launcher /Applications/IntelliJ IDEA2.app/Contents/lib/idea_rt.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/oro-2.0.8.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/resources_en.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/maven-model-3.6.1.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/qdox-2.0-M10.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/plexus-component-annotations-1.7.1.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/httpcore-4.4.13.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/maven-resolver-api-1.3.3.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/netty-common-4.1.47.Final.jar:/Applications/IntelliJ IDEA2.app/Contents/plugins/java/lib/maven-resolver-connector-basic-1.3.3.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/maven-artifact-3.6.1.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/plexus-utils-3.2.0.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/netty-resolver-4.1.47.Final.jar:/Applications/IntelliJ IDEA2.app/Contents/lib/guava-28.2-
40086 NativeOptimize
68879 RemoteMavenServer36
➜  jps -v
68848  -Xms128m -Xmx2048m -XX:ReservedCodeCacheSize=240m -XX:+UseCompressedOops -Dfile.encoding=UTF-8 -XX:+UseConcMarkSweepGC -XX:SoftRefLRUPolicyMSPerMB=50 -ea -XX:CICompilerCount=2 -Dsun.io.useCanonPrefixCache=false -Djava.net.preferIPv4Stack=true -Djdk.http.auth.tunneling.disabledSchemes="" -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Djdk.attach.allowAttachSelf -Dkotlinx.coroutines.debug=off -Djdk.module.illegalAccess.silent=true -Xverify:none -XX:ErrorFile=/Users/admin/java_error_in_idea_%p.log -XX:HeapDumpPath=/Users/admin/java_error_in_idea.hprof -javaagent:/Users/admin/.jetbrains/jetbrains-agent-v3.2.0.de72.619 -Djb.vmOptionsFile=/Users/admin/Library/Application Support/JetBrains/IntelliJIdea2020.1/idea.vmoptions -Didea.paths.selector=IntelliJIdea2020.1 -Didea.executable=idea -Didea.home.path=/Applications/IntelliJ IDEA2.app/Contents -Didea.vendor.name=JetBrains
40085 Launcher -Xmx700m -Djava.awt.headless=true -Djava.endorsed.dirs="" -Djdt.compiler.useSingleThread=true -Dpreload.project.path=/Users/admin/github/blog-example/blog-example -Dpreload.config.path=/Users/admin/Library/Application Support/JetBrains/IntelliJIdea2020.1/options -Dcompile.parallel=false -Drebuild.on.dependency.change=true -Djava.net.preferIPv4Stack=true -Dio.netty.initialSeedUniquifier=1366842080359982660 -Dfile.encoding=UTF-8 -Duser.language=zh -Duser.country=CN -Didea.paths.selector=IntelliJIdea2020.1 -Didea.home.path=/Applications/IntelliJ IDEA2.app/Contents -Didea.config.path=/Users/admin/Library/Application Support/JetBrains/IntelliJIdea2020.1 -Didea.plugins.path=/Users/admin/Library/Application Support/JetBrains/IntelliJIdea2020.1/plugins -Djps.log.dir=/Users/admin/Library/Logs/JetBrains/IntelliJIdea2020.1/build-log -Djps.fallback.jdk.home=/Applications/IntelliJ IDEA2.app/Contents/jbr/Contents/Home -Djps.fallback.jdk.version=11.0.6 -Dio.netty.noUnsafe=true -Djava.io.tmpdir=/Users/admin/Library/Caches/Je
40086 NativeOptimize -Dfile.encoding=UTF-8
40425 Jps -Dapplication.home=/Users/admin/Library/Java/JavaVirtualMachines/openjdk-14/Contents/Home -Xms8m -Djdk.module.main=jdk.jcmd
68879 RemoteMavenServer36 -Djava.awt.headless=true -Dmaven.defaultProjectBuilder.disableGlobalModelCache=true -Xmx768m -Didea.maven.embedder.version=3.6.1 -Dmaven.ext.class.path=/Applications/IntelliJ IDEA2.app/Contents/plugins/maven/lib/maven-event-listener.jar -Dfile.encoding=UTF-8

2. jstat (virtual machine statistics monitoring tool)

jstat (JVM Statistics Monitoring Tool) is used to monitor the running status of virtual machine.

For example, we use it to query the garbage collection of a Java process. The example is as follows:

➜  jstat -gc 43704
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT
10752.0 10752.0  0.0    0.0   65536.0   5243.4   175104.0     0.0     4480.0 774.0  384.0   75.8       0    0.000   0      0.000   -          -    0.000

The parameters are described in the following table:

Note: young gc will be triggered when the Edem area of the young generation is full, and old gc will be triggered when the old generation is full. full gc means to clear the whole heap, including young area and old area.

Common query parameters of jstat are:

  • -Class, query class loader information;
  • -compiler, JIT related information;
  • -GC, GC heap state;
  • -gcnew, statistical information of Cenozoic;
  • -gcutil, GC heap statistics summary.

3. jinfo (query virtual machine parameter configuration tool)

jinfo (Configuration Info for Java) is used to view and adjust parameters of virtual machine. The syntax is as follows:

jinfo
An example of JVM parameters is as follows:

➜ jinfo -flags 45129 VM Flags:
-XX:CICompilerCount=3 -XX:InitialHeapSize=268435456 -XX:MaxHeapSize=4294967296 -XX:MaxNewSize=1431306240
-XX:MinHeapDeltaBytes=524288 -XX:NewSize=89128960 -XX:OldSize=179306496 -XX:+UseCompressedClassPointers
-XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:+UseParallelGC

Among them, 45129 is LVMID using jps query.
We can modify the parameter value of the virtual machine through Jinfo - flag [+ /] name, such as the following example:

➜  jinfo -flag PrintGC 45129 # Query whether to turn on GC printing
-XX:-PrintGC
➜  jinfo -flag +PrintGC 45129 # Turn on GC printing
➜  jinfo -flag PrintGC 45129 # Query whether to turn on GC printing
-XX:+PrintGC
➜  jinfo -flag -PrintGC 45129 # Turn off GC printing
➜  jinfo -flag PrintGC 45129 # Query whether to turn on GC printing
-XX:-PrintGC

4. jmap (heap snapshot generation tool)

jmap (Memory Map for Java) is used to query the snapshot information of the heap.

An example of querying heap information is as follows:

➜  jmap -heap 45129
Attaching to process ID 45129, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.101-b13
using thread-local object allocation.
Parallel GC with 6 thread(s)
Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 4294967296 (4096.0MB)
   NewSize                  = 89128960 (85.0MB)
   MaxNewSize               = 1431306240 (1365.0MB)
   OldSize                  = 179306496 (171.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)
Heap Usage:
PS Young Generation
Eden Space:
   capacity = 67108864 (64.0MB)
   used     = 5369232 (5.1204986572265625MB)
   free     = 61739632 (58.87950134277344MB)
   8.000779151916504% used
From Space:
   capacity = 11010048 (10.5MB)
   used     = 0 (0.0MB)
   free     = 11010048 (10.5MB)
   0.0% used
To Space:
   capacity = 11010048 (10.5MB)
   used     = 0 (0.0MB)
   free     = 11010048 (10.5MB)
   0.0% used
PS Old Generation
   capacity = 179306496 (171.0MB)
   used     = 0 (0.0MB)
   free     = 179306496 (171.0MB)
   0.0% used
   2158 interned Strings occupying 152472 bytes.

We can also generate a heap snapshot file directly, for example:

➜ jmap -dump:format=b,file=/Users/admin/Documents/2020.dump 47380 Dumping heap to /Users/admin/Documents/2020.dump ... Heap dump file created

5. jhat (heap snapshot analysis function)

jhat (JVM Heap Analysis Tool) is used with jmap to start a web site to analyze the snapshot files generated by jmap.

The execution example is as follows:

jhat /Users/admin/Documents/2020.dump Reading from /Users/admin/Documents/2020.dump... Dump file created Tue May 26
16:12:41 CST 2020 Snapshot read, resolving... Resolving 17797 objects... Chasing references, expect 3 dots... Eliminating duplicate references... Snapshot resolved. Started HTTP server on port 7000 Server is ready.

The above information indicates that jhat has started a site with HTTP server port of 7000 to display the information. At this time, we enter: http://localhost:7000 /, you will see the following information:

6. jstack (query the current thread snapshot information of virtual machine)

jstack (Stack Trace for Java) is used to view the thread snapshot of the current virtual machine. It can be used to check the execution status of threads, such as deadlock, dead cycle, etc.

For example, let's write a deadlock code first:

public class NativeOptimize {
    private static Object obj1 = new Object();
    private static Object obj2 = new Object();
    public static void main(String[] args) {
        new Thread(new Runnable() {
            @Override
            public void run() {
                synchronized (obj2) {
                    System.out.println(Thread.currentThread().getName() + "lock up obj2");
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    synchronized (obj1) {
                        // We can't do it here
                        System.out.println("1 Seconds later," + Thread.currentThread().getName()
                                + "lock up obj1");
                    }
                }
            }
        }).start();
        synchronized (obj1) {
            System.out.println(Thread.currentThread().getName() + "lock up obj1");
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            synchronized (obj2) {
                // We can't do it here
                System.out.println("1 Seconds later," + Thread.currentThread().getName()
                        + "lock up obj2");
            }
        }
    }
}

The implementation results of the above procedures are as follows:

main: Lock obj 1
Thread-0: lock obp2

At this time, we use the jstack tool to print the snapshot information of the current thread. The results are as follows:

➜  bin jstack -l 50016
2020-05-26 18:01:41
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode):
"Attach Listener" #10 daemon prio=9 os_prio=31 tid=0x00007f8c00840800 nid=0x3c03 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"Thread-0" #9 prio=5 os_prio=31 tid=0x00007f8c00840000 nid=0x3e03 waiting for monitor entry [0x00007000100c8000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.example.optimize.NativeOptimize$1.run(NativeOptimize.java:25)
	- waiting to lock <0x000000076abb62d0> (a java.lang.Object)
	- locked <0x000000076abb62e0> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:745)
   Locked ownable synchronizers:
	- None
"Service Thread" #8 daemon prio=9 os_prio=31 tid=0x00007f8c01814800 nid=0x4103 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"C1 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007f8c0283c800 nid=0x4303 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007f8c0300a800 nid=0x4403 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007f8c0283c000 nid=0x3603 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007f8c0283b000 nid=0x4603 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   Locked ownable synchronizers:
	- None
"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007f8c03001000 nid=0x5003 in Object.wait() [0x000070000f8ad000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000076ab08ee0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
	- locked <0x000000076ab08ee0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   Locked ownable synchronizers:
	- None
"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007f8c03000000 nid=0x2f03 in Object.wait() [0x000070000f7aa000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000076ab06b50> (a java.lang.ref.Reference$Lock)
	at java.lang.Object.wait(Object.java:502)
	at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
	- locked <0x000000076ab06b50> (a java.lang.ref.Reference$Lock)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
   Locked ownable synchronizers:
	- None
"main" #1 prio=5 os_prio=31 tid=0x00007f8c00802800 nid=0x1003 waiting for monitor entry [0x000070000ef92000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.example.optimize.NativeOptimize.main(NativeOptimize.java:41)
	- waiting to lock <0x000000076abb62e0> (a java.lang.Object)
	- locked <0x000000076abb62d0> (a java.lang.Object)
   Locked ownable synchronizers:
	- None
"VM Thread" os_prio=31 tid=0x00007f8c01008800 nid=0x2e03 runnable
"GC task thread#0 (ParallelGC)" os_prio=31 tid=0x00007f8c00803000 nid=0x2007 runnable

"GC task thread#1 (ParallelGC)" os_prio=31 tid=0x00007f8c00006800 nid=0x2403 runnable

"GC task thread#2 (ParallelGC)" os_prio=31 tid=0x00007f8c01800800 nid=0x2303 runnable
"GC task thread#3 (ParallelGC)" os_prio=31 tid=0x00007f8c01801800 nid=0x2a03 runnable
"GC task thread#4 (ParallelGC)" os_prio=31 tid=0x00007f8c01802000 nid=0x5403 runnable
"GC task thread#5 (ParallelGC)" os_prio=31 tid=0x00007f8c01006800 nid=0x2d03 runnable
"VM Periodic Task Thread" os_prio=31 tid=0x00007f8c00010800 nid=0x3803 waiting on condition
JNI global references: 6
Found one Java-level deadlock:
=============================
"Thread-0":
  waiting to lock monitor 0x00007f8c000102a8 (object 0x000000076abb62d0, a java.lang.Object),
  which is held by "main"
"main":
  waiting to lock monitor 0x00007f8c0000ed58 (object 0x000000076abb62e0, a java.lang.Object),
  which is held by "Thread-0"

Java stack information for the threads listed above:
===================================================
"Thread-0":
	at com.example.optimize.NativeOptimize$1.run(NativeOptimize.java:25)
	- waiting to lock <0x000000076abb62d0> (a java.lang.Object)
	- locked <0x000000076abb62e0> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:745)
"main":
	at com.example.optimize.NativeOptimize.main(NativeOptimize.java:41)
	- waiting to lock <0x000000076abb62e0> (a java.lang.Object)
	- locked <0x000000076abb62d0> (a java.lang.Object)

Found 1 deadlock.

From the above information, we can see that using jstack can easily find out the problem of "deadlock" in the code.

Examination point analysis

The troubleshooting tool of Java virtual machine is a necessary skill for a qualified programmer. Using it, we can easily locate the problem. Especially in today's team cooperation, it's easy for everyone to have hidden bug s (defects). Therefore, using these troubleshooting functions can help us quickly locate and solve problems, so it is also one of the frequently asked questions in interviews.

The interview questions related to this knowledge point are as follows:

  • In addition to the more practical command-line tools, is there a more convenient troubleshooting tool?
  • What are the common tuning methods for JVM?

Knowledge expansion

Visual troubleshooting tool

In addition to the above six basic command-line tools, JVM also has two important view debugging tools, namely JConsole and JVisualVM. Compared with command-line tools, they are more convenient to use, easier to operate, and more intuitive to show the results.

Both JConsole and JVisualVM are located in the bin directory of JDK. JConsole (Java Monitoring and Management Console) is the earliest view debugging tool, and its startup page is shown as follows:


We can see that we can use it to connect to the remote server, or debug the local machine directly, so that we can start JConsole from the local machine to connect to the server without consuming the performance of the production environment. After the debugging process is selected, the running interface is as follows:

As can be seen from the above figure, JConsole can be used to monitor the related information of threads, CPU s, classes, heaps and VM S. Similarly, we can find the deadlock problem that we intentionally wrote before through the information on the thread page, as shown in the following figure:

You can see that the main and Thread-0 threads are deadlocked.

The startup diagram of JVisualVM is as follows:

It can be seen from the above figure that JVisualVM can debug both local and remote servers. When we select the relevant processes, the operation is as follows:

It can be seen that in addition to JConsole's information, JVisualVM has more details and is more intelligent. For example, this page of thread deadlock check is shown as follows:


It can be seen that JVisualVM will give you a deadlock prompt directly, while JConsole needs to be analyzed by the programmer himself.

JVM tuning

JVM tuning is mainly based on resetting JVM parameters according to the actual hardware configuration information. For example, the memory configuration of hardware is very high, but the JVM is the default parameter, so the maximum memory and initialization heap memory are very small, so it can not make better use of the advantages of local hardware. Therefore, these parameters need to be adjusted to maximize the value of the JVM in a fixed configuration.

Common JVM tuning parameters include the following:

  • -Xmx, set the maximum heap memory size;
  • -Xms, set the initial heap memory size;
  • -20: Maxnewsize, set the maximum memory of the new generation;
  • -20: Maxtenuringthreshold, set the object of the new generation to be promoted to the old generation after a certain number of times;
  • -20: Pretrnuresizethreshold, set the value of the large object. Objects exceeding this value will directly enter the old generation;
  • -20: Newratio, set the memory proportion of the new generation and the old generation of the generational garbage collector;
  • -20: Survivorratio, setting the proportion of new generation Eden, Form Survivor and To Survivor.

We need to set these values according to our own business scenarios and hardware configurations. For example, when many large temporary objects are generated in our business scenario, because these large objects have only a short life cycle, it is necessary to set the value of "- XX:MaxNewSize" as large as possible, otherwise a large number of large objects with short life cycle will enter the old generation, which will quickly consume the memory of the old generation, so that full will be triggered frequently GC, which affects the normal operation of the business.

Summary

In this lesson, we talked about six basic command-line tools for JVM Troubleshooting: jps, jstat, jinfo, jmap, jhat, jstack, and two view troubleshooting tools: JConsole and JVisualVM. At the same time, we talked about common JVM tuning parameters, hoping that the content of this lesson can help you.

Tags: Java jvm snapshot Maven

Posted on Tue, 23 Jun 2020 04:31:08 -0400 by Jewbilee