Detailed solution based on Java memory overflow
- 2020-04-01 01:58:15
- OfStack
One, memory overflow type
1, Java. Lang. OutOfMemoryError: PermGen space
The JVM manages two types of memory, heap and non-heap. The heap is for the developer which is created when the JVM starts; The non-heap is left to the JVM itself to hold information about the class. Unlike the heap, the GC does not free up space during runtime. If the web app USES a large number of third-party jars or the application has too many class files and the MaxPermSize is set to be too small, it will also lead to excessive use of this memory and overflow. Or when tomcat is hot deployed, it will not clean up the previously loaded environment and only change the context to the newly deployed one, and the non-stackable content will become more and more.
The full name of PermGen space is the Permanent Generation space, which refers to the Permanent storage area of memory. This memory is mainly stored by the JVM for Class and Meta information. Class will be put into PermGen space when loaded. So if you have a CLASS in your application, you are likely to have a PermGen space error, which is common when a web server pre compiles a JSP. This error message is generated if your WEB APP USES a large number of third-party jars that exceed the JVM's default size (4M).
An example of the best configuration :(I have verified that there has been no tomcat death since I used this configuration)
Set JAVA_OPTS= -xms800m-xmx800m-xx :PermSize= 128m-xx :MaxNewSize= 256m-xx :MaxPermSize=256m
2, Java. Lang. OutOfMemoryError: Java heap space
The first case is a supplement, and the main problem is in this case. The default space (i.e. -xms) is 1/64 of physical memory, and the maximum space (-xmx) is 1/4 of physical memory. If less than 40% of memory is left, the JVM increases the heap to the Xmx setting, and if more than 70% of memory is left, the JVM decreases the heap to the Xms setting. So the server's Xmx and Xms Settings should generally be set the same to avoid having to resize the virtual machine heap after each GC. Assuming that the physical memory is infinite, the maximum value of the JVM's memory depends on the operating system. Typically, 32-bit machines are between 1.5 gb and 3g, while 64-bit machines are unlimited.
Note: if Xms exceeds the Xmx value, or the sum of the heap maximum and non-heap maximum exceeds the physical memory or operating system maximum, the server will not start.
The role of the garbage collection GC
The frequency of GC calls by the JVM is still high, and there are two main scenarios for garbage collection:
When application threads are idle; The other is that when Java runs out of memory, the GC is called repeatedly, and if successive collections fail to solve the problem, an out of memory error is reported. Because this exception is determined by the system environment, it is impossible to predict when it will occur.
According to the GC mechanism, the running of the program will cause changes in the running environment of the system, increasing the chance of GC triggering.
To avoid these problems, programs should be designed and written to avoid the memory footprint of garbage objects and the overhead of GC. It shows that calling system.gc () only suggests that the JVM needs to recycle garbage objects in memory, but not right away,
One is that it does not solve the problem of memory resource depletion, and it also increases GC consumption.
Ii. JVM memory region composition
Simply speak of stacks and stacks in Java
Java divides memory into two types: stack memory and heap memory
1. The basic type variables defined in the function and the reference variables of the object are allocated in the stack memory of the function.
2. Heap memory is used to hold objects and arrays created by new
When a variable is defined in a function (code block), Java allocates memory space for the variable in the stack. When the scope of the variable is exceeded, Java automatically frees up the memory space allocated for the variable. The memory allocated in the heap is managed by the Java virtual machine's automatic garbage collector
The advantage of the heap is that you can dynamically allocate memory sizes, and you don't have to tell the compiler in advance about the lifetime, because it dynamically allocates memory at run time. The disadvantage is to dynamically allocate memory at run time, access speed is slow;
The advantage of the stack is that the access speed is faster than the heap, the disadvantage is that the data size and lifetime in the stack must be fixed without flexibility.
The Java heap is divided into three sections: New, Old, and Permanent
GC has two threads:
Newly created objects are allocated to the New area, which is moved to the Old area by the GC worker when the area is filled, and when the Old area is also filled it triggers the GC main thread to traverse all the objects in the heap. The size of the Old block is equal to Xmx minus minus Xmn
Java stack storage
Stack adjustment: the parameter is + usedefaultstacksize-xss256k, which means each thread can apply for 256k stack space
Each thread has its own Stack
How does the JVM set up virtual memory
Tip: this exception information is thrown in the JVM if 98% of the time is spent in GC and the available Heap size is less than 2%.
Tip: the Heap Size should not exceed 80% of the available physical memory, and the -xms and -xmx options should generally be set to the same, while -xmn is 1/4 of the -xmx value.
Tip: the JVM's initial allocation of memory is specified by -xms, which defaults to 1/64 of physical memory. The maximum allocated memory for the JVM is specified by -xmx, which defaults to 1/4 of physical memory.
When the default free heap memory is less than 40%, the JVM increases the heap up to the maximum limit of -xmx. When the free heap memory is greater than 70%, the JVM reduces the heap up to the minimum limit of -xms. So the server typically sets -xms and -xmx equal to avoid resizing the heap after each GC.
Tip: assuming that physical memory is infinite, the maximum amount of memory in the JVM has a lot to do with the operating system.
Simply put, 32-bit processors have 4GB of controllable memory space, but specific operating systems have a limit,
This limit is generally 2gb-3gb (generally 1.5g-2g for Windows and 2gg-3g for Linux), and no limit is imposed on processors above 64bit
Tip: note: if Xms exceeds the Xmx value, or the sum of the heap maximum and non-heap maximum exceeds the physical memory or operating system maximum, the server will not start.
Tip: set the size of NewSize and MaxNewSize to be the same, and the size of "new" should not be more than half of the size of "old". The reason is that if the old area is not large enough to trigger the "main" GC frequently, the performance will be greatly reduced
The JVM USES -xx: PermSize to set the initial value of non-heap memory, which is 1/64 of physical memory by default.
Set the maximum size of non-heap memory by XX:MaxPermSize, which is 1/4 of physical memory by default.
Solution: set the Heap size manually
Modify the TOMCAT_HOME/bin/catalina. Bat
Add the following line above "echo "Using CATALINA_BASE: $CATALINA_BASE" :
JAVA_OPTS = "- server - Xms800m - Xmx800m - XX: MaxNewSize = 256 m"
Use of performance checking tools
Locate memory leaks:
The JProfiler tool is primarily used to check and track the performance of systems (limited to Java development). Jprofilers can monitor JVM performance and performance by constantly monitoring the system's memory usage, garbage collection, thread running, etc.
1. The memory of the application server is occupied for a long time, and the memory is often occupied at a high level, so it is difficult to recover to a low level;
2. The application server is extremely unstable and is restarted almost every two days, sometimes even every day;
3. The application server often does Full GC(Garbage Collection), and it takes a long time, about 30-40 seconds. When the application server does Full GC, it does not respond to the customer's transaction request, which greatly affects the system performance.
Because of the development environment and production environment will have different, caused the problems sometimes happen in the production environment, you can often use tool tracking system memory usage, in some individual cases may sometime really is to use a large amount of memory lead to out of memory, then should continue to track to see whether there will be next fall,
If it stays high it must be a memory leak because of the program.
Five, not robust code features and solutions
1. Release references to unwanted objects early. A good idea is to have the reference variable automatically set to null after exiting the active domain when using a temporary variable, hinting the garbage collector to collect the object and preventing a memory leak.
For instances that still have Pointers to them, the JVM will not recycle the resource, because garbage collection will garbage the null object, improving the efficiency of the GC collection mechanism.
2. It is inevitable to use a lot of String processing in our program, so we should avoid using String. Instead, we should use a lot of StringBuffer.
String STR = "aaa".
String str2 = "BBB";
String str3 = STR + str2; // if STR and str2 are not called again after this execution, it will be placed in memory waiting for the gc of Java to recover. If too many such cases occur in the program, the error will be reported.
3, as far as possible to use less static variables, because static variables are global, GC will not recover;
4. Avoid the centralized creation of objects, especially large objects. The JVM will suddenly need a large amount of memory. Displays the declared array space, and the number of requests is also very large.
This is a case for your vigilance
Using jspsmartUpload for file upload, java.outofMemoryError often occurs during the operation.
Check and find the problem: the code in the component
M_totalBytes = m_request. GetContentLength ();
M_binArray = new byte [m_totalBytes];
The reason for the problem is that the variable totalBytes gets a large number, which causes the array to allocate a lot of memory space, and the array cannot be released in time. The solution can only be a more appropriate solution, at least one that does not cause an outofMemoryError.
5. Try to use object pool technology to improve system performance; When objects with long life cycle have objects with short life cycle, memory leak is easy to be caused. For example, when a large collection object has a large amount of business objects with large data, it can be considered to divide the processing into blocks, and then solve the strategy of releasing one block by one block.
Don't create objects in frequently invoked methods, especially in loops. With hashtable, vector can be used appropriately to create a set of object containers and then fetch those objects from the container, rather than throwing them away after each new
7. Generally, it occurs when a large file is opened or too much data is taken from a database at one time, resulting in an Out Of Memory Error. At this time, it is necessary to calculate the maximum value Of data and set the minimum and maximum value Of Memory space required.
1, Java. Lang. OutOfMemoryError: PermGen space
The JVM manages two types of memory, heap and non-heap. The heap is for the developer which is created when the JVM starts; The non-heap is left to the JVM itself to hold information about the class. Unlike the heap, the GC does not free up space during runtime. If the web app USES a large number of third-party jars or the application has too many class files and the MaxPermSize is set to be too small, it will also lead to excessive use of this memory and overflow. Or when tomcat is hot deployed, it will not clean up the previously loaded environment and only change the context to the newly deployed one, and the non-stackable content will become more and more.
The full name of PermGen space is the Permanent Generation space, which refers to the Permanent storage area of memory. This memory is mainly stored by the JVM for Class and Meta information. Class will be put into PermGen space when loaded. So if you have a CLASS in your application, you are likely to have a PermGen space error, which is common when a web server pre compiles a JSP. This error message is generated if your WEB APP USES a large number of third-party jars that exceed the JVM's default size (4M).
An example of the best configuration :(I have verified that there has been no tomcat death since I used this configuration)
Set JAVA_OPTS= -xms800m-xmx800m-xx :PermSize= 128m-xx :MaxNewSize= 256m-xx :MaxPermSize=256m
2, Java. Lang. OutOfMemoryError: Java heap space
The first case is a supplement, and the main problem is in this case. The default space (i.e. -xms) is 1/64 of physical memory, and the maximum space (-xmx) is 1/4 of physical memory. If less than 40% of memory is left, the JVM increases the heap to the Xmx setting, and if more than 70% of memory is left, the JVM decreases the heap to the Xms setting. So the server's Xmx and Xms Settings should generally be set the same to avoid having to resize the virtual machine heap after each GC. Assuming that the physical memory is infinite, the maximum value of the JVM's memory depends on the operating system. Typically, 32-bit machines are between 1.5 gb and 3g, while 64-bit machines are unlimited.
Note: if Xms exceeds the Xmx value, or the sum of the heap maximum and non-heap maximum exceeds the physical memory or operating system maximum, the server will not start.
The role of the garbage collection GC
The frequency of GC calls by the JVM is still high, and there are two main scenarios for garbage collection:
When application threads are idle; The other is that when Java runs out of memory, the GC is called repeatedly, and if successive collections fail to solve the problem, an out of memory error is reported. Because this exception is determined by the system environment, it is impossible to predict when it will occur.
According to the GC mechanism, the running of the program will cause changes in the running environment of the system, increasing the chance of GC triggering.
To avoid these problems, programs should be designed and written to avoid the memory footprint of garbage objects and the overhead of GC. It shows that calling system.gc () only suggests that the JVM needs to recycle garbage objects in memory, but not right away,
One is that it does not solve the problem of memory resource depletion, and it also increases GC consumption.
Ii. JVM memory region composition
Simply speak of stacks and stacks in Java
Java divides memory into two types: stack memory and heap memory
1. The basic type variables defined in the function and the reference variables of the object are allocated in the stack memory of the function.
2. Heap memory is used to hold objects and arrays created by new
When a variable is defined in a function (code block), Java allocates memory space for the variable in the stack. When the scope of the variable is exceeded, Java automatically frees up the memory space allocated for the variable. The memory allocated in the heap is managed by the Java virtual machine's automatic garbage collector
The advantage of the heap is that you can dynamically allocate memory sizes, and you don't have to tell the compiler in advance about the lifetime, because it dynamically allocates memory at run time. The disadvantage is to dynamically allocate memory at run time, access speed is slow;
The advantage of the stack is that the access speed is faster than the heap, the disadvantage is that the data size and lifetime in the stack must be fixed without flexibility.
The Java heap is divided into three sections: New, Old, and Permanent
GC has two threads:
Newly created objects are allocated to the New area, which is moved to the Old area by the GC worker when the area is filled, and when the Old area is also filled it triggers the GC main thread to traverse all the objects in the heap. The size of the Old block is equal to Xmx minus minus Xmn
Java stack storage
Stack adjustment: the parameter is + usedefaultstacksize-xss256k, which means each thread can apply for 256k stack space
Each thread has its own Stack
How does the JVM set up virtual memory
Tip: this exception information is thrown in the JVM if 98% of the time is spent in GC and the available Heap size is less than 2%.
Tip: the Heap Size should not exceed 80% of the available physical memory, and the -xms and -xmx options should generally be set to the same, while -xmn is 1/4 of the -xmx value.
Tip: the JVM's initial allocation of memory is specified by -xms, which defaults to 1/64 of physical memory. The maximum allocated memory for the JVM is specified by -xmx, which defaults to 1/4 of physical memory.
When the default free heap memory is less than 40%, the JVM increases the heap up to the maximum limit of -xmx. When the free heap memory is greater than 70%, the JVM reduces the heap up to the minimum limit of -xms. So the server typically sets -xms and -xmx equal to avoid resizing the heap after each GC.
Tip: assuming that physical memory is infinite, the maximum amount of memory in the JVM has a lot to do with the operating system.
Simply put, 32-bit processors have 4GB of controllable memory space, but specific operating systems have a limit,
This limit is generally 2gb-3gb (generally 1.5g-2g for Windows and 2gg-3g for Linux), and no limit is imposed on processors above 64bit
Tip: note: if Xms exceeds the Xmx value, or the sum of the heap maximum and non-heap maximum exceeds the physical memory or operating system maximum, the server will not start.
Tip: set the size of NewSize and MaxNewSize to be the same, and the size of "new" should not be more than half of the size of "old". The reason is that if the old area is not large enough to trigger the "main" GC frequently, the performance will be greatly reduced
The JVM USES -xx: PermSize to set the initial value of non-heap memory, which is 1/64 of physical memory by default.
Set the maximum size of non-heap memory by XX:MaxPermSize, which is 1/4 of physical memory by default.
Solution: set the Heap size manually
Modify the TOMCAT_HOME/bin/catalina. Bat
Add the following line above "echo "Using CATALINA_BASE: $CATALINA_BASE" :
JAVA_OPTS = "- server - Xms800m - Xmx800m - XX: MaxNewSize = 256 m"
Use of performance checking tools
Locate memory leaks:
The JProfiler tool is primarily used to check and track the performance of systems (limited to Java development). Jprofilers can monitor JVM performance and performance by constantly monitoring the system's memory usage, garbage collection, thread running, etc.
1. The memory of the application server is occupied for a long time, and the memory is often occupied at a high level, so it is difficult to recover to a low level;
2. The application server is extremely unstable and is restarted almost every two days, sometimes even every day;
3. The application server often does Full GC(Garbage Collection), and it takes a long time, about 30-40 seconds. When the application server does Full GC, it does not respond to the customer's transaction request, which greatly affects the system performance.
Because of the development environment and production environment will have different, caused the problems sometimes happen in the production environment, you can often use tool tracking system memory usage, in some individual cases may sometime really is to use a large amount of memory lead to out of memory, then should continue to track to see whether there will be next fall,
If it stays high it must be a memory leak because of the program.
Five, not robust code features and solutions
1. Release references to unwanted objects early. A good idea is to have the reference variable automatically set to null after exiting the active domain when using a temporary variable, hinting the garbage collector to collect the object and preventing a memory leak.
For instances that still have Pointers to them, the JVM will not recycle the resource, because garbage collection will garbage the null object, improving the efficiency of the GC collection mechanism.
2. It is inevitable to use a lot of String processing in our program, so we should avoid using String. Instead, we should use a lot of StringBuffer.
String STR = "aaa".
String str2 = "BBB";
String str3 = STR + str2; // if STR and str2 are not called again after this execution, it will be placed in memory waiting for the gc of Java to recover. If too many such cases occur in the program, the error will be reported.
3, as far as possible to use less static variables, because static variables are global, GC will not recover;
4. Avoid the centralized creation of objects, especially large objects. The JVM will suddenly need a large amount of memory. Displays the declared array space, and the number of requests is also very large.
This is a case for your vigilance
Using jspsmartUpload for file upload, java.outofMemoryError often occurs during the operation.
Check and find the problem: the code in the component
M_totalBytes = m_request. GetContentLength ();
M_binArray = new byte [m_totalBytes];
The reason for the problem is that the variable totalBytes gets a large number, which causes the array to allocate a lot of memory space, and the array cannot be released in time. The solution can only be a more appropriate solution, at least one that does not cause an outofMemoryError.
5. Try to use object pool technology to improve system performance; When objects with long life cycle have objects with short life cycle, memory leak is easy to be caused. For example, when a large collection object has a large amount of business objects with large data, it can be considered to divide the processing into blocks, and then solve the strategy of releasing one block by one block.
Don't create objects in frequently invoked methods, especially in loops. With hashtable, vector can be used appropriately to create a set of object containers and then fetch those objects from the container, rather than throwing them away after each new
7. Generally, it occurs when a large file is opened or too much data is taken from a database at one time, resulting in an Out Of Memory Error. At this time, it is necessary to calculate the maximum value Of data and set the minimum and maximum value Of Memory space required.