Bug 44389 - Cleanup of org.apache.naming.resources.CacheEntry resources (memory leak)
Summary: Cleanup of org.apache.naming.resources.CacheEntry resources (memory leak)
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 6
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 6.0.14
Hardware: PC All
: P2 normal (vote)
Target Milestone: default
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-02-11 02:03 UTC by Diego Rodriguez
Modified: 2008-04-16 14:56 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Diego Rodriguez 2008-02-11 02:03:09 UTC
Hi,

   I am memory profiling a webapp for my company because we suspect it has
memory leaks when redeploying. This webapp has to be redeployed several times a
week for security reasons (most of the time it is undeployed) and the memory
leaks forced us to shutdown tomcat once a week to avoid OOM exception.

   I'm using jvm1.5 and tomcat 6.0.13 (also tried with 6.0.14 and today with
6.0.16), and Yourkit as the profiling tool. I have made the test in both linux
and windows.

   I have created a JMeter test to deploy the app using the manager, make some
request to the app and then undeploy it using again the manager app in a loop.
After some redeployments, I got my OutOfMemoryException. Then I checked with my
profiling tool and found that my own classes were correctly destroyed, and the
classes retaining more memory were these ones (listed as a heirarchy):

   org.apache.catalina.core.ApplicationContext$DispatchData
   org.apache.catalina.core.ApplicationContext
mapper of  org.apache.catalina.core.StandardContext
context of  org.apache.tomcat.util.http.mapper.Mapper
resources of  org.apache.tomcat.util.http.mapper.Mapper$Context  cache of 
org.apache.naming.resources.ProxyDirContext  cache of 
org.apache.naming.resources.ResourceCache
org.apache.naming.resources.CacheEntry[]

   The number of objects of each class were exactly the number of times I had
redeployed the webapp. I have checked the contents of CacheEntry inside
CacheEntry[] and it has an attribute "name" and the contents are the jsps and
tld files of my app. There are as many CacheEntry with the same jsp attribute as
times the webapp has been reloaded, so I guess there is no cleaning of
CacheEntry resources at context shutdown.

   To reproduce this, a simple webapp with 2 jsp and a taglibrary can be used.
Deploy and undeploy it many times so you can see how the count of these objects
grows in memory.
Comment 1 Arto Huusko 2008-03-04 05:26:16 UTC
Caused by non-static inner class DispatchData in org.apache.cataline.core.ApplicationContext being held in a (non-static) ThreadLocal.

While seemingly dispatchData is GC'able, Sun JVM Thread class holds reference to the ThreadLocal object through Thread class. Example path (from jhat):

--> java.lang.ThreadGroup@0x2aa4b85608 (67 bytes)  (field threads:)
--> [Ljava.lang.Thread;@0x2aa674dda0 (272 bytes) (Element 21 of [Ljava.lang.Thread;@0x2aa674dda0:)
--> java.lang.Thread@0x2aa6f4b418 (164 bytes) (field threadLocals:)
--> java.lang.ThreadLocal$ThreadLocalMap@0x2aa6f86d18 (32 bytes) (field table:)
--> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x2aa6f86d38 (528 bytes) (Element 16 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x2aa6f86d38:)
--> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x2aa6f87408 (56 bytes) (field value:)
--> org.apache.catalina.core.ApplicationContext$DispatchData@0x2aa6f87440 (40 bytes) (field this$0:)
--> org.apache.catalina.core.ApplicationContext@0x2aa6f87b88 (72 bytes)


Because dispatchData field is non-static, what ever it contains is leaked when a particular ApplicationContext is discarded: a new ThreadLocal object is created for each ApplicationContext instance in each Thread, and when the instance is discarded, an unaccessible but still not GC'able reference stays in the JVM's Thread instances that used it.

Because DispatchData inner class is not static, it also causes a reference to the ApplicationContext instance itself, which leads to catastrophic results, if the ClassLoader of the web app is referencable throught the ApplicationContext.
Comment 2 Mark Thomas 2008-04-11 15:36:54 UTC
Testing with YourKit confirms that making the nested class static fixed the leak.

Thanks for the analysis.

The fix has been applied to trunk and proposed for 6.0.x.
Comment 3 Mark Thomas 2008-04-16 14:56:37 UTC
This has been fixed in svn for 6.0.x and will be included in 6.0.17 onwards.