|Summary:||Cleanup of org.apache.naming.resources.CacheEntry resources (memory leak)|
|Product:||Tomcat 6||Reporter:||Diego Rodriguez <drodriguez>|
|Component:||Catalina||Assignee:||Tomcat Developers Mailing List <dev>|
Description Diego Rodriguez 2008-02-11 02:03:09 UTC
Hi, I am memory profiling a webapp for my company because we suspect it has memory leaks when redeploying. This webapp has to be redeployed several times a week for security reasons (most of the time it is undeployed) and the memory leaks forced us to shutdown tomcat once a week to avoid OOM exception. I'm using jvm1.5 and tomcat 6.0.13 (also tried with 6.0.14 and today with 6.0.16), and Yourkit as the profiling tool. I have made the test in both linux and windows. I have created a JMeter test to deploy the app using the manager, make some request to the app and then undeploy it using again the manager app in a loop. After some redeployments, I got my OutOfMemoryException. Then I checked with my profiling tool and found that my own classes were correctly destroyed, and the classes retaining more memory were these ones (listed as a heirarchy): org.apache.catalina.core.ApplicationContext$DispatchData org.apache.catalina.core.ApplicationContext mapper of org.apache.catalina.core.StandardContext context of org.apache.tomcat.util.http.mapper.Mapper resources of org.apache.tomcat.util.http.mapper.Mapper$Context cache of org.apache.naming.resources.ProxyDirContext cache of org.apache.naming.resources.ResourceCache org.apache.naming.resources.CacheEntry The number of objects of each class were exactly the number of times I had redeployed the webapp. I have checked the contents of CacheEntry inside CacheEntry and it has an attribute "name" and the contents are the jsps and tld files of my app. There are as many CacheEntry with the same jsp attribute as times the webapp has been reloaded, so I guess there is no cleaning of CacheEntry resources at context shutdown. To reproduce this, a simple webapp with 2 jsp and a taglibrary can be used. Deploy and undeploy it many times so you can see how the count of these objects grows in memory.
Comment 1 Arto Huusko 2008-03-04 05:26:16 UTC
Caused by non-static inner class DispatchData in org.apache.cataline.core.ApplicationContext being held in a (non-static) ThreadLocal. While seemingly dispatchData is GC'able, Sun JVM Thread class holds reference to the ThreadLocal object through Thread class. Example path (from jhat): --> java.lang.ThreadGroup@0x2aa4b85608 (67 bytes) (field threads:) --> [Ljava.lang.Thread;@0x2aa674dda0 (272 bytes) (Element 21 of [Ljava.lang.Thread;@0x2aa674dda0:) --> java.lang.Thread@0x2aa6f4b418 (164 bytes) (field threadLocals:) --> java.lang.ThreadLocal$ThreadLocalMap@0x2aa6f86d18 (32 bytes) (field table:) --> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x2aa6f86d38 (528 bytes) (Element 16 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x2aa6f86d38:) --> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x2aa6f87408 (56 bytes) (field value:) --> org.apache.catalina.core.ApplicationContext$DispatchData@0x2aa6f87440 (40 bytes) (field this$0:) --> org.apache.catalina.core.ApplicationContext@0x2aa6f87b88 (72 bytes) Because dispatchData field is non-static, what ever it contains is leaked when a particular ApplicationContext is discarded: a new ThreadLocal object is created for each ApplicationContext instance in each Thread, and when the instance is discarded, an unaccessible but still not GC'able reference stays in the JVM's Thread instances that used it. Because DispatchData inner class is not static, it also causes a reference to the ApplicationContext instance itself, which leads to catastrophic results, if the ClassLoader of the web app is referencable throught the ApplicationContext.
Comment 2 Mark Thomas 2008-04-11 15:36:54 UTC
Testing with YourKit confirms that making the nested class static fixed the leak. Thanks for the analysis. The fix has been applied to trunk and proposed for 6.0.x.
Comment 3 Mark Thomas 2008-04-16 14:56:37 UTC
This has been fixed in svn for 6.0.x and will be included in 6.0.17 onwards.