Bug 56390 - Tomcat keeps jar files in <app>/WEB-INF/lib opened and app cannot be fully undeployed
Summary: Tomcat keeps jar files in <app>/WEB-INF/lib opened and app cannot be fully un...
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 8.0.x-trunk
Hardware: PC All
: P2 normal (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2014-04-11 13:04 UTC by Violeta Georgieva
Modified: 2015-03-23 12:45 UTC (History)
1 user (show)

Profiling analyses (394.17 KB, image/jpeg)
2014-04-11 13:04 UTC, Violeta Georgieva
example (360.05 KB, application/octet-stream)
2014-04-11 15:48 UTC, Violeta Georgieva

Note You need to log in before you can comment on or make changes to this bug.
Description Violeta Georgieva 2014-04-11 13:04:03 UTC
Created attachment 31513 [details]
Profiling analyses


If there are jar files in <app>/WEB-INF/lib, Tomcat keeps them opened and during undeploy they cannot be cleared. The following error appears:

"SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.startup.ExpandWar.delete [...] could not be completely deleted. The presence of the remaining files may cause problems" 

The problem is because during startup of the application, a call to o.a.catalina.loader.WebappClassLoader.findResource is made, which opens the files. See the profiling analyses that are attached. (openedfiles.jpg)

A possible solution would be to invoke java.net.URLClassLoader.close() in org.apache.catalina.loader.WebappClassLoader.stop method:

Index: C:/tc8.0.x/java/org/apache/catalina/loader/WebappClassLoader.java
--- C:/tc8.0.x/java/org/apache/catalina/loader/WebappClassLoader.java	(revision 1585681)
+++ C:/tc8.0.x/java/org/apache/catalina/loader/WebappClassLoader.java	(working copy)
@@ -1497,6 +1497,12 @@
+        try {
+            super.close();
+        } catch (IOException e) {
+            throw new LifecycleException(e);
+        }
What do you think?

Comment 1 Mark Thomas 2014-04-11 13:15:59 UTC
Steps to reproduce?
Comment 2 Sebb 2014-04-11 13:46:40 UTC
May not be directly relevant, but I've noticed that Java on Windows keeps an OS lock on files that it has got open - even when read-only. So one cannot delete jars that are in use (and it makes tailing log files tricky in Windows!)
Unix Java does not seem to have this restriction.
Comment 3 Konstantin Kolinko 2014-04-11 15:06:16 UTC
> A possible solution would be to invoke java.net.URLClassLoader.close() in
> org.apache.catalina.loader.WebappClassLoader.stop method:

1. What will be an effect of calling stop(),start() (I have not checked if anything calls those methods in a pair).

If URLClassLoader cannot recover after a call to close() then it should be done in destroy() instead of stop().

2. Are you running without a JreMemoryLeakPreventionListener ?
Comment 4 Violeta Georgieva 2014-04-11 15:47:16 UTC
(In reply to Mark Thomas from comment #1)
> Steps to reproduce?

- ant clean deploy
- go to output/build and start Tomcat
- deploy the attached application
- undeploy the application
Comment 5 Violeta Georgieva 2014-04-11 15:48:27 UTC
Created attachment 31515 [details]
Comment 6 Violeta Georgieva 2014-04-11 15:50:40 UTC
(In reply to Konstantin Kolinko from comment #3)
> 2. Are you running without a JreMemoryLeakPreventionListener ?

It is there I did a regular "ant clean deploy" without any additional changes
Comment 7 Mark Thomas 2014-04-13 02:00:05 UTC
Looking at this with YourKit the proposed patch (modified as suggested by Konstantin) addresses 2 out of the 4 open file issues I am seeing with lsof.

I want to see if I can find the source of the remaining 2 files before I commit a fix.

Note that triggering GC does clean up all the remaining open files.
Comment 8 Mark Thomas 2014-04-16 14:23:12 UTC
I've committed a partial fix for this. I made some progress with tracking down the cause of the other files locks but still have some work to do there.
Comment 9 Mark Thomas 2014-04-16 18:00:15 UTC
Fix in 8.0.x for 8.0.6 onwards.
Comment 10 Dennis Lundberg 2014-08-29 12:42:44 UTC
We are seeing this issue as well in Tomcat 6.0.35 on Windows 7.
Is there any chance of a back-port to Tomcat 6 or 7 for this fix?