Bug 57772 - WebappClassLoader throws a ClassNotFoundError when the Manager is deploying a new WAR
Summary: WebappClassLoader throws a ClassNotFoundError when the Manager is deploying a...
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: Manager (show other bugs)
Version: 8.0.21
Hardware: PC Linux
: P2 critical (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-03-27 18:53 UTC by Austin Jones
Modified: 2015-04-01 18:35 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Austin Jones 2015-03-27 18:53:32 UTC
Overview:
When contexts are updated through the Tomcat manager interface, ServletContextListener.contextDestroyed implementations which need to load classes throw ClassNotFoundError.

Steps to reproduce:
1. Create an instance of Tomcat 8.0.20
2. In server.xml, configure the Host with unpackWARs="true" autoDeploy="false".
3. Check out the example webapp from https://github.com/austinjones/ClassNotFoundGenerator
4. Build the testing WAR using the 'distribute' ant task.
5. Deploy the WAR using an HTTP request to this URL:
http://<your-local-tomcat>/manager/text/deploy?
 war=path/to/workspace/ClassNotFoundGenerator/dist/ClassNotFoundGenerator.war
 &config=path/to/workspace/ClassNotFoundGenerator/web/context.xml
 &path=/ClassNotFoundGenerator
 &update=true
6. Open localhost.log, and verify the ClassNotFoundError was logged.

Actual results:
The ClassNotFoundError is thrown by TestcaseContextListener.contextDestroyed, and printed to localhost.log.  Any further work the context listener was responsible for is not executed.

Expected Results:
The WebappClassLoader successfully loads classes during ServletContextListener.contextDestroyed, and completes without a throw.  No ClassNotFoundError is printed to localhost.log

Build Date & Hardware:
Tomcat 8.0.12 on CentOS Linux 6.6

Additional Builds and Platforms:
Tomcat 8.0.12 on Windows 7 SP1
Tomcat 8.0.20 on Windows 7 SP1
Tomcat 8.0.20 on CentOS Linux 6.6

Note: if you try the 'steps to reproduce' on 8.0.12, the deployment will probably fail due to bug 56398 - which I worked around in 8.0.12.  You'll be able to deploy if you change the name of the test app to 'classnotfoundgenerator'.

Additional Information:
ServletContextListeners that need to load classes during the contextDestroyed call throw ClassNotFoundError, when the application is deployed as a WAR through the Manager interface.  Here is an example listener, where ThisClassNotFound is not loaded for the first time in contextDestroyed.
  https://github.com/austinjones/ClassNotFoundGenerator/blob/master/src/com/avadyne/TestcaseContextListener.java

The stack trace of the ClassNotFoundError on 8.0.20 is:
6-Mar-2015 14:38:46.838 SEVERE [http-nio-8443-exec-5] org.apache.catalina.core.StandardContext.listenerStop Exception sending context destroyed event to listener instance of class com.avadyne.TestcaseContextListener
 java.lang.NoClassDefFoundError: com/avadyne/ThisClassNotFound
	at com.avadyne.TestcaseContextListener.contextDestroyed(TestcaseContextListener.java:27)
	at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4775)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5385)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232)
	at org.apache.catalina.core.StandardContext.reload(StandardContext.java:3739)
	at org.apache.catalina.startup.HostConfig.reload(HostConfig.java:1304)
	at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1236)
	at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1491)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:300)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
	at org.apache.catalina.manager.ManagerServlet.check(ManagerServlet.java:1460)
	at org.apache.catalina.manager.ManagerServlet.deploy(ManagerServlet.java:906)
	at org.apache.catalina.manager.ManagerServlet.doGet(ManagerServlet.java:344)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:618)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:613)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:516)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1086)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:659)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1558)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1515)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.avadyne.ThisClassNotFound
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1305)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1157)
	... 45 more

The stack trace where Tomcat attempts to load ThisClassNotFound is (8.0.20):
Daemon Thread [http-nio-8443-exec-31] (Suspended)	
	owns: WebappClassLoader  (id=3483)	
	owns: TestcaseContextListener  (id=3488)	
	owns: StandardContext  (id=3324)	
	owns: HostConfig  (id=3323)	
	owns: SecureNioChannel  (id=3313)	
	StandardRoot.getResourceInternal(String, boolean) line: 302	
	Cache.getResource(String, boolean) line: 65	
	StandardRoot.getResource(String, boolean, boolean) line: 216	
	StandardRoot.getClassLoaderResource(String) line: 225	
	WebappClassLoader(WebappClassLoaderBase).findResourceInternal(String, String) line: 2548	
	WebappClassLoader(WebappClassLoaderBase).findClassInternal(String) line: 2405	
	WebappClassLoader(WebappClassLoaderBase).findClass(String) line: 854	
	WebappClassLoader(WebappClassLoaderBase).loadClass(String, boolean) line: 1274	
	WebappClassLoader(WebappClassLoaderBase).loadClass(String) line: 1157	
	TestcaseContextListener.contextDestroyed(ServletContextEvent) line: 27	
	StandardContext.listenerStop() line: 4775	
	StandardContext.stopInternal() line: 5385	
	StandardContext(LifecycleBase).stop() line: 232	
	StandardContext.reload() line: 3739	
	HostConfig.reload(HostConfig$DeployedApplication) line: 1304	
	HostConfig.checkResources(HostConfig$DeployedApplication) line: 1236	
	HostConfig.check(String) line: 1491	
	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
	Method.invoke(Object, Object...) line: 483	
	BaseModelMBean.invoke(String, Object[], String[]) line: 300	
	DefaultMBeanServerInterceptor.invoke(ObjectName, String, Object[], String[]) line: 819	
	JmxMBeanServer.invoke(ObjectName, String, Object[], String[]) line: 801	
	ManagerServlet.check(String) line: 1460	
	ManagerServlet.deploy(PrintWriter, String, ContextName, String, boolean, StringManager) line: 906	
	ManagerServlet.doGet(HttpServletRequest, HttpServletResponse) line: 344	
	ManagerServlet(HttpServlet).service(HttpServletRequest, HttpServletResponse) line: 618	
	ManagerServlet(HttpServlet).service(ServletRequest, ServletResponse) line: 725	
	ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 291	
	ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206	
	WsFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 52	
	ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 239	
	ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206	
	SetCharacterEncodingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 108	
	ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 239	
	ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206	
	StandardWrapperValve.invoke(Request, Response) line: 219	
	StandardContextValve.invoke(Request, Response) line: 106	
	BasicAuthenticator(AuthenticatorBase).invoke(Request, Response) line: 613	
	StandardHostValve.invoke(Request, Response) line: 142	
	ErrorReportValve.invoke(Request, Response) line: 79	
	AccessLogValve(AbstractAccessLogValve).invoke(Request, Response) line: 610	
	StandardEngineValve.invoke(Request, Response) line: 88	
	CoyoteAdapter.service(Request, Response) line: 516	
	Http11NioProcessor(AbstractHttp11Processor).process(SocketWrapper<S>) line: 1086	
	Http11NioProtocol$Http11ConnectionHandler(AbstractProtocol$AbstractConnectionHandler).process(SocketWrapper<S>, SocketStatus) line: 659	
	Http11NioProtocol$Http11ConnectionHandler.process(SocketWrapper<NioChannel>, SocketStatus) line: 223	
	NioEndpoint$SocketProcessor.doRun(SelectionKey, NioEndpoint$KeyAttachment) line: 1558	
	NioEndpoint$SocketProcessor.run() line: 1515	
	ThreadPoolExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142	
	ThreadPoolExecutor$Worker.run() line: 617	
	TaskThread$WrappingRunnable.run() line: 61	
	TaskThread(Thread).run() line: 745	

The ClassLoader gives up on loading the class.  The WebResourceSet it would have used to load the class from (in the method StandardRoot.getResourceInternal) is a DirResourceSet pointed to the exploded directory path - e.g. /path/to/tomcat/webapps/ClassNotFoundGenerator/.  The path it generates in DirResourceSet.getResource is /path/to/tomcat/webapps/ClassNotFoundGenerator/WEB-INF/classes/com/avadyne/ThisClassNotFound.class - the path is correct.  The file doesn't exist, and it returns a new EmptyResource(root, path, f).

When the WebappClassLoader attempts to load the class, the .class file (and the entire exploded directory) is not on the disk.  The bug occurs during these frames of the ClassNotFoundError stack trace (8.0.20).

 java.lang.NoClassDefFoundError: com/avadyne/ThisClassNotFound
...
 	at org.apache.catalina.core.StandardContext.reload(StandardContext.java:3739)
	at org.apache.catalina.startup.HostConfig.reload(HostConfig.java:1304)
	at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1236)
...

HostConfig.checkResources:1231 deletes the exploded directory
HostConfig.checkResources:1236 calls into StandardContext.reload
StandardContext.reload:3739 eventually invokes the ServletContextListeners

Once the contextDestroyed listeners are executed, the .class file doesn't exist (HostConfig.checkResources deleted it).  

It is unsafe for HostConfig.checkResources to delete the unpacked directory before the context is stopped.  However, WebappClassLoader is correct to throw a ClassNotFoundError, since the class doesn't exist on disk.
Comment 1 Austin Jones 2015-03-27 18:54:29 UTC
There is additional information in the Tomcat users mailing list thread entitled 'ClassNotFoundError on context unload'.
Comment 2 Chuck Caldarale 2015-03-27 19:10:57 UTC
(In reply to Austin Jones from comment #1)
> There is additional information in the Tomcat users mailing list thread
> entitled 'ClassNotFoundError on context unload'.

The thread referred to is here:
http://marc.info/?l=tomcat-user&m=142697641505022&w=2
Comment 3 Austin Jones 2015-03-27 21:53:15 UTC
Sorry, the 8.0.12 case issue is bug 56938.
Comment 4 Christopher Schultz 2015-03-28 01:58:14 UTC
IIRC, this also affects 8.0.21.
Comment 5 Mark Thomas 2015-03-28 09:04:27 UTC
Thanks for the analysis. I see what is going on and why. In theory, the fix is simple: don't delete the directory until the app has been stopped. In practice, the cleanest way to fix this needs some thought.
Comment 6 Mark Thomas 2015-04-01 17:10:07 UTC
Fixed in trunk, 8.0.x (for 8.0.22 onwards) and 7.0.x (for 7.0.62 onwards).

Thanks for the report, the analysis and the test case.
Comment 7 Austin Jones 2015-04-01 18:35:53 UTC
(In reply to Mark Thomas from comment #6)
> Thanks for the report, the analysis and the test case.

No problem.  Thank you for fixing this so quickly!