In the follow-up, I will continuously update the source code, principle and introduction of the virtual machine. If you think it's useful to yourself, pay attention.
Most of the content of this article is extracted from Zhou Zhiming, in-depth understanding of Java virtual machine: JVM advanced features and best practices (3rd Edition)
And add some of my own understanding and reference materials
first, understand the difference between memory overflow and memory leak:
- Memory leak: due to some operations of developers, the memory that should have been recovered has not been recovered.
- Memory overflow: simply available memory is not enough.
Set IDEA virtual machine parameters
-Xxs20m / / set the minimum value of virtual machine heap
-Xmx20m / / set the maximum value of virtual machine heap
-20: + heapdumponoutofmemoryerror / / enables the virtual machine to Dump the current memory heap Dump snapshot in case of memory overflow exception for post analysis
Code example
import java.util.ArrayList; import java.util.List; /** * @author: wangxu * @date: 2021-10-10 14:04 */ public class JvmTest { public static void main(String[] args) { List<String> list = new ArrayList<>(); while (true){ list.add("Site string"); } } }
Operation results:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3210) at java.util.Arrays.copyOf(Arrays.java:3181) at java.util.ArrayList.grow(ArrayList.java:265) at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239) at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231) at java.util.ArrayList.add(ArrayList.java:462) at JvmTest.main(JvmTest.java:12)
Notice the Java heap space prompted at the end of the log
Treatment method
in this case, we must first judge whether it is a memory overflow or a memory leak.
- Memory leak: in this case, we need to check the reference chain from the leaked object to GC Roots through the corresponding memory image analysis tools (there are many such tools, you can check them), and find out the reference path and GC Roots associated with the leaked object, so that the garbage collector cannot recycle them, According to the type information of the leaked object and its reference chain to GC Roots, you can generally accurately locate the location where these objects are created, and then find out the specific location of the code causing memory leakage.
- Memory overflow: when this happens, it means that all objects are necessary, so we can only adjust the parameter of the maximum heap memory of the virtual machine, increase its value, or optimize the corresponding algorithm.
HostSpot virtual machine does not distinguish between virtual machine stack and local method stack. Therefore, for HostSpot, although the - Xoss parameter (setting the size of local method stack) exists, it has no effect. The stack capacity can only be set by the - Xss parameter.
for virtual machine stack and local method stack overflow, two exceptions are described in the Java virtual machine specification:
- If the stack depth requested by the thread is greater than the maximum depth allowed by the virtual machine, a StackOverflowError exception will be thrown.
- If the stack memory of the virtual machine allows dynamic expansion, an OutOfMemoryError exception is thrown when the extended stack capacity cannot apply for enough memory.
the Java virtual machine specification explicitly allows the Java virtual machine to choose whether to support the dynamic expansion of the virtual machine stack, while HotSpot chooses not to support the expansion. Therefore, unless the OutOfMemoryError exception occurs when the thread is created to apply for memory because it cannot obtain enough memory, the memory overflow will not be caused by the expansion when the thread is running, The StackOverflowError exception is only caused because the stack capacity cannot accommodate new stack frames
Set IDEA virtual machine parameters
-Xss128k / / set the size of the virtual machine stack to 128k
Example code
The first case (the request stack depth is too large)
/** * @author: wangxu * @date: 2021-10-10 14:04 */ public class JvmTest { private int stackLength = 1; public void checkTest(){ stackLength++; checkTest(); } public static void main(String[] args) { JvmTest jvmTest = new JvmTest(); try { jvmTest.checkTest(); }catch (Throwable e){ System.out.println("Stack depth:" + jvmTest.stackLength); throw e; } } }
this is an infinite recursive program. Of course, the program can't let you recurse indefinitely. After reaching a certain stack depth, the system will return an error! The following is the error message.
Stack depth:990 Exception in thread "main" java.lang.StackOverflowError at JvmTest.checkTest(JvmTest.java:11) at JvmTest.checkTest(JvmTest.java:12) at JvmTest.checkTest(JvmTest.java:12) at JvmTest.checkTest(JvmTest.java:12) at JvmTest.checkTest(JvmTest.java:12) (The following error reports are the same, so 10000 words are omitted here!)
for different versions of Java virtual machines and different operating systems, the minimum stack capacity may be limited, which mainly depends on the memory page size of the operating system. For example, the parameter - Xss128k in the above method can be normally used for JDK 6 under 32-bit Windows system. However, if it is used for JDK 11 under 64 bit Windows system, it will prompt that the minimum stack capacity cannot be less than 180K, while the value may be 228K under Linux. If it is lower than this minimum limit, the following prompt will be given when the HotSpot virtual machine starts:
The Java thread stack size specified is too small. Specify at least 228k
The second case (insufficient memory allocated for thread method frames)
first, we tried to increase the memory space of each stack frame, so we decided to add a large number of local variables to the method to increase the length of the local variable table in the method frame.
/** * @author: wangxu * @date: 2021-10-10 14:04 */ public class JvmTest { private static int stackLength = 0; public static void test() { long unused1, unused2, unused3, unused4, unused5, unused6, unused7, unused8, unused9, unused10, unused11, unused12, unused13, unused14, unused15, unused16, unused17, unused18, unused19, unused20, unused21, unused22, unused23, unused24, unused25, unused26, unused27, unused28, unused29, unused30, unused31, unused32, unused33, unused34, unused35, unused36, unused37, unused38, unused39, unused40, unused41, unused42, unused43, unused44, unused45, unused46, unused47, unused48, unused49, unused50, unused51, unused52, unused53, unused54, unused55, unused56, unused57, unused58, unused59, unused60, unused61, unused62, unused63, unused64, unused65, unused66, unused67, unused68, unused69, unused70, unused71, unused72, unused73, unused74, unused75, unused76, unused77, unused78, unused79, unused80, unused81, unused82, unused83, unused84, unused85, unused86, unused87, unused88, unused89, unused90, unused91, unused92, unused93, unused94, unused95, unused96, unused97, unused98, unused99, unused100; stackLength ++; test(); unused1 = unused2 = unused3 = unused4 = unused5 = unused6 = unused7 = unused8 = unused9 = unused10 = unused11 = unused12 = unused13 = unused14 = unused15 = unused16 = unused17 = unused18 = unused19 = unused20 = unused21 = unused22 = unused23 = unused24 = unused25 = unused26 = unused27 = unused28 = unused29 = unused30 = unused31 = unused32 = unused33 = unused34 = unused35 = unused36 = unused37 = unused38 = unused39 = unused40 = unused41 = unused42 = unused43 = unused44 = unused45 = unused46 = unused47 = unused48 = unused49 = unused50 = unused51 = unused52 = unused53 = unused54 = unused55 = unused56 = unused57 = unused58 = unused59 = unused60 = unused61 = unused62 = unused63 = unused64 = unused65 = unused66 = unused67 = unused68 = unused69 = unused70 = unused71 = unused72 = unused73 = unused74 = unused75 = unused76 = unused77 = unused78 = unused79 = unused80 = unused81 = unused82 = unused83 = unused84 = unused85 = unused86 = unused87 = unused88 = unused89 = unused90 = unused91 = unused92 = unused93 = unused94 = unused95 = unused96 = unused97 = unused98 = unused99 = unused100 = 0; } public static void main(String[] args) { try { test(); }catch (Error e){ System.out.println("stack length:" + stackLength); throw e; } } }
Operation results
stack length:51 Exception in thread "main" java.lang.StackOverflowError at JvmTest.test(JvmTest.java:32) at JvmTest.test(JvmTest.java:33) at JvmTest.test(JvmTest.java:33) at JvmTest.test(JvmTest.java:33) at JvmTest.test(JvmTest.java:33)
The third case (the number of threads is too large and the stack memory is insufficient when allocating new threads)
in fact, the memory allocated to each process in the computer is several. For example, in a 32-bit system, the maximum memory allocated to each process is 2GB. In this way, we calculate a formula:
Virtual machine stack and local method stack = 2GB - maximum heap memory capacity - maximum method area memory capacity - program counter memory capacity (this is very small and can be ignored) - internal memory - memory consumed by the virtual machine process itself
because the stack is thread private, if a thread is not created, it will allocate a share of memory. When the remaining memory is allocated, the next thread will report an OOM exception!
/** * VM Args: -Xss2M (At this time, you might as well set it larger, please run it under the 32-bit system) * @author: wangxu * @date: 2021-10-10 14:04 */ public class JvmTest { private static int stackLength = 0; public static void dontStop() { while(true){ } } public static void main(String[] args) { while(true){ Thread thread = new Thread(new Runnable() { @Override public void run() { dontStop(); } }); thread.start(); } } }
Operation results
Exception in thread "main" java.lang.OutOfMemoryError: unable to create native threadMethod area and runtime constant pool overflow
let's talk about this part briefly, because in the version after Java 7, HostSpot has removed the "permanent generation" of the method area and replaced it with the meta space in the heap. Next, we will simply see several overflow in the method area caused by the permanent generation
Example code
/** * VM Args: -XX:PermSize=6M -XX:MaxPermSize=6M * @author wangxu */ public class RuntimeConstantPoolOOM { public static void main(String[] args) { // Use Set to maintain constant pool reference and avoid Full GC reclaiming constant pool Set<String> set = new HashSet<String>(); // In the short range, it is enough for 6MB PermSize to generate OOM short i = 0; while (true) { set.add(String.valueOf(i++).intern()); } } }
Operation results:
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space at java.lang.String.intern(Native Method) at org.fenixsoft.oom.RuntimeConstantPoolOOM.main(RuntimeConstantPoolOOM.java: 18)
This is an exception in Java 6. Java 8 will not throw such an exception because the string constant pool originally stored in the permanent generation is moved to the Java heap.
This paragraph is more important. I suggest reading it
let's look at other parts of the method area. The main responsibility of the method area is to store type related information, such as class name, access modifier, constant pool, field description, method description, etc. For the testing of this part of the area, the basic idea is to generate a large number of classes at runtime to fill the method area until it overflows. Although classes can also be generated dynamically by using Java SE API directly (such as GeneratedConstructorAccessor and dynamic proxy during reflection), it is troublesome to operate in this experiment. In code listing 2-9, the author directly operates bytecode with the help of CGLib, and generates a large number of dynamic classes at runtime.
it is worth noting that the scenario we simulated in this example is not purely an experiment. Codes like this may indeed appear in practical applications: many current mainstream frameworks, such as Spring and Hibernate, use bytecode technologies such as CGLib when enhancing classes. When more classes are enhanced, The larger the method area is needed to ensure that dynamically generated new types can be loaded into memory. In addition, many dynamic languages running on the Java virtual machine (such as Groovy) usually continue to create new types to support the dynamics of the language. With the popularity of such dynamic languages, overflow scenarios similar to those in code listing 2-9 are becoming more and more easy to encounter.
method area overflow is also a common memory overflow exception. If a class is to be recycled by the garbage collector, the conditions to be met are relatively harsh. In application scenarios where a large number of dynamic classes are often generated at runtime, special attention should be paid to the recycling of these classes. In addition to the previously mentioned programs using CGLib bytecode enhancement and dynamic language, common scenarios include: a large number of JSPS or applications that dynamically generate JSP files (JSPS need to be compiled into Java classes when running for the first time), OSGi based applications (even if the same class file is loaded by different loaders, it will be regarded as different classes), etc
after JDK 8, the permanent generation completely withdrew from the historical stage and meta space came on as its substitute. Under the default setting, the normal dynamic creation of new types of test cases listed above can hardly force the virtual machine to generate overflow exceptions in the method area. However, in order for users to prevent destructive operations similar to those in code listing 2-9 in practical applications, HotSpot still provides some parameters as defense measures for meta space, mainly including:
· - XX: MaxMetaspaceSize: sets the maximum value of the meta space, which is - 1 by default, that is, it is not limited, or only limited to the local memory size.
· - XX: MetaspaceSize: Specifies the initial space size of the meta space, in bytes. Reaching this value will trigger garbage collection for type unloading. At the same time, the collector will adjust this value: if a large amount of space is released, it is appropriate
Reduce the value; If a small amount of space is released, increase the value appropriately without exceeding - XX: MaxMetaspaceSize (if set).
· - XX: MinMetaspaceFreeRatio: it is used to control the percentage of the minimum remaining meta space after garbage collection, which can reduce the frequency of garbage collection caused by insufficient meta space. Similarly, there is - XX: Max metaspacefreeratio, which is used to control the percentage of the maximum remaining Metaspace capacity.
the capacity of direct memory can be determined by:
- -XX: MaxDirectMemorySize
If it is not specified, it is consistent with the maximum value of Java heap (specified by - Xmx) by default
Let's not give an example. Let's talk about the example in the book, which is to use NIO to repeatedly apply for memory locally, and know that the application reaches the maximum position. A pit is buried in this position, which will be published together with NIO articles in the future.