On reload of a application memory usage in the PermGen rises steadily as does the loaded classes count. If you reload and access the application many times and the remove the application and take a memory dump you'll that there are still many instances of the WebappClassLoader in memory. If you then look for what refers to these instances you find cross classloader references. A specific case is that I find the ApplicationDispatcher class which is loaded by StandardClassLoader has a static reference to an instance of org.apache.commons.logging.impl.Log4JLogger that was loaded by one of the WebappClassLoader s. I have a very small war that I've been able to reproduce this problem with which I'll attach. I think the important bits are that the application includes log4j and commons-logging in the WEB-INF/lib directory and some how some catalina classes are seeing and using this class. I've tested this with both Sun Java Sun 1.6.0 and 1.6.0_01.
Created attachment 20000 [details] WAR to recreate the problem
Tomcat will use the logging configuration of your web application for logging related to the application. As a result, this may not work out that well if you're also using reloading. In that case, I would recommend deploying your application differently (log4j could be moved out of /WEB-INF/lib, for example).
I just want to make sure I fully understand this. If a application uses log4j and commons logging then that application can not be redeployed without a full stop/start of the server? The work around you mention is moving log4j and I assume commons-logging out of the /WEB-INF/lib. Where would be the best place to locate it then? Shared, Common or someplace elese?
I've been looking at this further and I really think there's something wrong here. What I find wrong comes down to the fact that the variable causing the cross reference (ApplicationDispatcher.log) is a static. How does a static variable allow for the app server to do per webapp logging. So I created 2 identical webapps. They have a single servlet that simply dispatches to a JSP. I added code to the dispatch that looks at the classloader for the servlet, request dispatcher and the log of the request dispatcher. I then hit webapp 1 and then webapp2. I reloaded both apps and did the same again. The code and output is included below. Basically the ApplicationDispatcher has a log loaded by the classloader for the first webapp I hit. After a reload the log is still the log loaded by that first webapp classloader and that classloader is marked not started. In addition, the parent instance variable defined in WebappClassLoader is null. I'm not particularly knowledgeable about Tomcat yet but it would seem that using classes loaded from one webapp while servicing the request for another webapp is bad. And continuing to use classes loaded by a classloader that's been stopped would seem a very bad idea. =========== Code inserted into the servlet =================== ServletContext sc = null; RequestDispatcher rd = null; sc = getServletContext(); rd = sc.getRequestDispatcher(uri); Object log = getField(rd, "log"); ClassLoader loaderC = rd.getClass().getClassLoader(); System.out.println(loaderC); ClassLoader loaderA = this.getClass().getClassLoader(); System.out.println(loaderA); ClassLoader loaderB = log.getClass().getClassLoader(); System.out.println(loaderB); System.out.println("servlet loader " + (loaderA == loaderB ? "equals" : "does not equal") + " RequestDispatcher's log loader"); System.out.println("servlet loader " + (loaderA == loaderC ? "equals" : "does not equal") + " RequestDispatcher loader"); System.out.println("RequestDispatcher's log loader " + (loaderB == loaderC ? "equals" : "does not equal") + " RequestDispatcher loader"); ================= Output immediately after the server comes up ======= WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 servlet loader equals RequestDispatcher's log loader servlet loader does not equal RequestDispatcher loader RequestDispatcher's log loader does not equal RequestDispatcher loader org.apache.catalina.loader.StandardClassLoader@5e55ab WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 servlet loader does not equal RequestDispatcher's log loader servlet loader does not equal RequestDispatcher loader RequestDispatcher's log loader does not equal RequestDispatcher loader ============== Output after reloading both of the applications ======= org.apache.catalina.loader.StandardClassLoader@5e55ab WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 WebappClassLoader delegate: false repositories: servlet loader does not equal RequestDispatcher's log loader servlet loader does not equal RequestDispatcher loader RequestDispatcher's log loader does not equal RequestDispatcher loader org.apache.catalina.loader.StandardClassLoader@5e55ab WebappClassLoader delegate: false repositories: /WEB-INF/classes/ ----------> Parent Classloader: org.apache.catalina.loader.StandardClassLoader@31f2a7 WebappClassLoader delegate: false repositories: servlet loader does not equal RequestDispatcher's log loader servlet loader does not equal RequestDispatcher loader RequestDispatcher's log loader does not equal RequestDispatcher loader
This bug and 41939 may be related.
You have every right to find it wrong if you like, and I perfectly understand the "issue", however what I am telling you is that this will not be fixed. As far as I am concerned, providing the logging framework with a webapp is possible, and can sometimes be useful, but reloading will cause problems. If you want to use reloading, you can either stop using log4j and use java.util.logging instead (as it is a one-per-JVM logging service, it is much more robust), or package your application differently (log4j and the commons-logging wrapper for log4j should be moved to a shared CL repository). Reopening this report is not going to change this.
Re-opening to change resolution.
I have tracked down three places where the logger for a class was causing a memory leak on reload with the provided test case. These have been fixed in svn and will be included in 5.5.24 onwards. Many thanks for the test case, it made tracking things down much easier.
Any component can keep a reference to a logger, so it just means the logging setup is inapropriate with respect to the intended usage. Attemting to fix this is not going to provide any benefit.
This has now been fixed for the provided test case but other use cases may continue to see similar issues.