This is the old issue of log4j trying to use the previous classloader (reloading
will create a new classloader to load class definitions; see bug 3888). Maybe it
would be good to leave this bug open so that people can complain using it,
rather than file duplicates. However, I'd like everyone to know that the "bug"
will never be fixed.
You can probably fix the problem by putting log4 JARs (and the necessary
commons-logging wrapper classes) inside the webapp repository.

The problem has long been fixed by adding LogManager.shutdown() the
contextDestroyed() method of a servlet context listener.  I have mentioned this
before, but for some reason no one takes notice.  I'm writing it here so that at
least there is a reference.

Jake

The suggested fix of putting the log4j jars in WEB-INF/lib doesn't seem to work.

We are getting this a lot after recompilation of a class in a web app which 
makes Tomcat reload the application. 

However, we DO have the LogManager.shutdown in our servlet context listener but 
it does not solve the problem. 

	public void contextDestroyed(ServletContextEvent scE) {		
		LogManager.shutdown();  
	}

Interestingly this has only occurred the past 2 weeks since using 5.0.25. Prior 
to this we developed for 7 months with no problem on 5.0.19. So we just tried 
rolling back to 5.0.19 but it ALSO started happening there too. We find this 
behaviour very very odd. Furthermore we never had a LogManager.shutdown for 
those 7 months either. There is a lot of conflicting behaviour going on here 
and I would really like to know how to solve this as right now we are looking 
at having to rip out log4j which is bad news. 

SEVERE: Exception invoking periodic operation: 
java.lang.ThreadDeath
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1229)
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1189)
	at java.lang.ClassLoader.loadClassInternal(Unknown Source)
	at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:241)
	at org.apache.log4j.Category.forcedLog(Category.java:431)
	at org.apache.log4j.Category.log(Category.java:966)
	at org.apache.commons.logging.impl.Log4JLogger.error
(Log4JLogger.java:195)
	at org.apache.catalina.session.StandardManager.start
(StandardManager.java:659)
	at org.apache.catalina.core.StandardContext.start
(StandardContext.java:4272)
	at org.apache.catalina.core.StandardContext.reload
(StandardContext.java:3021)
	at org.apache.catalina.core.StandardContext.backgroundProcess
(StandardContext.java:4629)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1619)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1628)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1628)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run
(ContainerBase.java:1608)
	at java.lang.Thread.run(Unknown Source)

Relates to : http://issues.apache.org/bugzilla/show_bug.cgi?id=27371

Per Jay Paulsen's findings archived at http://marc.theaimsgroup.com/?
t=109578393000004&r=1&w=2, please try adding the Introspector.flush call and re-
running the test.

Unless I hear that the combination of solutions offered in this thread does not 
address this problem, I will close this issue as WORKFORME soon.

A couple of questions:

1) Does it actually work for you?
2) Does it work for Allistair Crossley ?
3) Wouldn't closing the report hide or obfuscate the information contained herein? 

I'm on it. Will let you know ASAP what the outcome of Introspector.flushCaches
() is on this bug.

Can confirm that the ThreadDeath is still occurring after every class compile 
that causes a webapp reload by Tomcat resulting in total failure of Tomcat with 
the following in our contextDestroyed

LogManager.shutdown();
java.beans.Introspector.flushCaches();
LogFactory.releaseAll();

Allistair,

Do you observe any difference when you configure log4j only by using 
BasisConfigurator.configure()? I mean without calling PropertyConfigurator or 
DOMConfigruator...

i use log4j only by placing log4j.properties into classes in my web 
application, not programmatically as I think you are suggesting.

> i use log4j only by placing log4j.properties into classes in my web 
> application, not programmatically as I think you are suggesting.

The log4j.properties file is parsed by PropertiesConfigurator. So you are indirectly 
invoking PropertiesConfigurator. 

Do you observe any difference if you remove log4j.properties file, i.e. not 
configure log4j at all? What do you observe if you call BasicConfigurator.configure() 
method?

can confirm that removal of log4j.properties from web application allows Tomcat 
to reload webapp gracefully after class compilation with dynamic class 
reloading on.

Two questions:

1) Could you include the contents of your log4j.properties file.
2) Are you 100% sure that the trio
  - LogManager.shutdown();
  - java.beans.Introspector.flushCaches();
  - LogFactory.releaseAll();java.beans.Introspector.flushCaches() 
really gets called?

OK, 

Can confirm that contextDestroyed is called. Discovered LogManager.shutdown(); 
is working because a logging statement afterwards does not materialize but one 
before it is fine. System.out.println was used instead. Here is the relevant 
logging

INFO: Reloading this Context has started
before Renewals Application Destroyed
one Renewals Application Destroyed
two Renewals Application Destroyed
after Renewals Application Destroyed
log4j:WARN No appenders could be found for logger 
(org.apache.struts.util.PropertyMessageResources).
log4j:WARN Please initialize the log4j system properly.
05-Oct-2004 16:13:38 org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already 
(the eventual following stack trace is caused by an error thrown for debugging 
purposes as well as to attempt to terminate the thread which caused the illegal 
access, and has no functional impact)
05-Oct-2004 16:13:38 org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already 
(the eventual following stack trace is caused by an error thrown for debugging 
purposes as well as to attempt to terminate the thread which caused the illegal 
access, and has no functional impact)
05-Oct-2004 16:13:38 org.apache.commons.modeler.Registry registerComponent
SEVERE: Null component 
Catalina:type=JspMonitor,WebModule=//localhost/,J2EEApplication=none,J2EEServer=
none
05-Oct-2004 16:13:38 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor 
processChildren
SEVERE: Exception invoking periodic operation: 
java.lang.ThreadDeath
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1229)
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1189)
	at java.lang.ClassLoader.loadClassInternal(Unknown Source)
	at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:241)
	at org.apache.log4j.Category.forcedLog(Category.java:431)
	at org.apache.log4j.Category.log(Category.java:966)
	at org.apache.commons.logging.impl.Log4JLogger.error
(Log4JLogger.java:195)
	at org.apache.catalina.session.StandardManager.start
(StandardManager.java:659)
	at org.apache.catalina.core.StandardContext.start
(StandardContext.java:4272)
	at org.apache.catalina.core.StandardContext.reload
(StandardContext.java:3021)
	at org.apache.catalina.core.StandardContext.backgroundProcess
(StandardContext.java:4629)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1619)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1628)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChild
ren(ContainerBase.java:1628)
	at 
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run
(ContainerBase.java:1608)
	at java.lang.Thread.run(Unknown Source)

And here is the log4j.properties file.

################################################################################
#
# Renewals Log4j Configuration
#
################################################################################

################################################################################
#### Root
################################################################################

#log4j.appender.stdout=org.apache.log4j.ConsoleAppender
#log4j.appender.stdout.Target=System.out
#log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
#log4j.appender.stdout.layout.ConversionPattern=%d - %5p (%C:%L) - %m%n

log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=c:/jakarta-tomcat-5.0.25/logs/root.log
log4j.appender.R.MaxFileSize=100KB
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d - %5p (%C:%L) - %m%n

log4j.rootCategory=error, R

################################################################################
#### Named
################################################################################

log4j.logger.Application=debug, ApplicationA
log4j.appender.ApplicationA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.ApplicationA.Threshold=DEBUG
log4j.appender.ApplicationA.ImmediateFlush=true
log4j.appender.ApplicationA.File=c:/jakarta-tomcat-5.0.25/logs/sys-
application.log
log4j.appender.ApplicationA.Append=true
log4j.appender.ApplicationA.layout=org.apache.log4j.PatternLayout
log4j.appender.ApplicationA.layout.ConversionPattern=%d - %5p (%C:%L) - %m%n
log4j.appender.ApplicationA.DatePattern='.'yyyy-MM-dd

log4j.logger.Errors=debug, ErrorsA
log4j.appender.ErrorsA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.ErrorsA.Threshold=DEBUG
log4j.appender.ErrorsA.ImmediateFlush=true
log4j.appender.ErrorsA.File=c:/jakarta-tomcat-5.0.25/logs/sys-errors.log
log4j.appender.ErrorsA.Append=true
log4j.appender.ErrorsA.layout=org.apache.log4j.PatternLayout
log4j.appender.ErrorsA.layout.ConversionPattern=%d - %5p (%C:%L) - %m%n
log4j.appender.ErrorsA.DatePattern='.'yyyy-MM-dd

################################################################################
#### Classes
################################################################################

#log4j.logger.net.sf.hibernate=debug, ApplicationA
#log4j.logger.net.sf.hibernate.type=debug, ApplicationA
#log4j.logger.net.sf.hibernate.tool.hbm2ddl=debug, ApplicationA

################################################################################

log4j.logger.org.apache.struts=debug, R

I suppose the following output lines confim that the trio we talked abour earlier is 
being called. 

INFO: Reloading this Context has started
before Renewals Application Destroyed
one Renewals Application Destroyed
two Renewals Application Destroyed
after Renewals Application Destroyed

Am I Correct?

What happens if you simplify the log4j.properties file to say:

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d - %5p (%C:%L) - %m%n
log4j.rootCategory=error, console
log4j.logger.org.apache.struts=debug

If you wish, we can continue this discussion directly. My email is Ceki AATT qos 
DDOOTT ch.

You are correct. These were just output messages entered between the 3 calls to 
ensure they were being called.

We can confirm ThreadDeath occurs even with the simplified log4j.

Cheers.

Ceki asked for log4j and commons-logging jar locations. It revealed a disparity 
between commons-logging in tomcat's bin and the webapp lib.

Initial tests with our original log4j file and now replacing commons-logging in 
the webapp with the commons-logging-api version found in tomcat's bin, is 
successful.

I suppose this will disappear from TC 5.5 but it does appear to have gone away 
for now ..

We will keep an eye on it, but this appears to be the problem

I wouldn't be quick to conclude this will disappear from Tomcat 5.5.  Tomcat 
5.5. still uses commons-logging (in fact, more heavily than before), and still 
locates the commons-logging jar on the bin classpath, because it uses it from 
the very startup of the server.  Tomcat 5.0 and 5.5 both require commons-
logging.  What your investigation reveals, I think, is that the version of 
commons-logging used by the webapp must be the same as that uses by Tomcat 
(which uses the latest stable commons-logging build).

I've commented a while ago on how to properly package commons-logging:
- commons-logging-api goes in the system classloader (it will go first in the
delegation order)
- put the necessary commons-logging wrapper classes for the logger you're using
at the same spot as your logger JAR
This seemed to avoid problems.

If you (any of the people following this thread) want, please suggest text 
that I can put in the following places:
- Tomcat's Logging configuration page
- Tomcat's FAQ
- Log4J FAQ
- Other places you find appropriate

Otherwise, I'll come up with some text myself (which will include a link to 
this Bugzilla issue) and place the text at the above locations.  At that time 
I will close this item, as it's not a Tomcat bug and there's no reason for it 
to stay open.  

I have found out that the reason I managed to have this problem in the first 
place is because I upgraded to Struts 1.2.4 recently where the bundled JAR for 
common is commons-logging.jar. Would it make sense to rename Tomcat's JAR to 
commons-logging? Also, is it really not a Tomcat problem that user's cannot use 
APIs like Struts and who knows what else from Jakarta that demand or come 
packaged with commons-logging? It seems to me that the webapps should work 
first and foremost with whatever APIs they like in their LIB and should not 
have to worry about the application server dying?

There's a significant difference between commons-logging.jar and commons-
logging-api.jar, it's not just a rename.  Every version of commons-logging has 
both distributions and for good reason.

As for the rant about being able to use whatever library and not wortying about 
the web server dying: I'd point out it's only 'dying' when you're trying the 
app reload, nor normal usage.  This feature is not mandated by the Spec so 
we're not obliged to provided it in the first place: it's caused mostly trouble 
and has very limited usage in production environments.  So if you have patches 
for it, that's great, but the common usage scenario for Tomcat doesn't include 
webapp reload, and so doesn't suffer from this issue at all.

sorry you thought it was a rant, it wasn't meant to come over like that :) i 
suppose because tomcat has a manager application and reloadable class facility, 
i assumed it should do that gracefully in all circumstances. it is true i don't 
fully appreciate all the issues with this feature so i suppose this is just an 
issue we have to workaround. In fact, the workaround has been listed here now.

It is also true that this is in our case was always related to a development 
instance. 

I am interested in your comment that reload of webapp is mostly trouble and 
limited in production environments though. My understanding was that if you 
want to make a build to production or a patch of some files, you would use ant 
or similar as we do here to reconstruct the WAR to deploy. Does this not 
require tomcat being able to reload? In fact, we tell the business the intranet 
will be down for 5 minutes and post a message page up for inbound requests. We 
stop tomcat, delete the old war and expanded war files and place the new war 
and startup tomcat again. We constantly get irked by the fact that if a bug is 
on production we have to wait until the evening to patch it whereas our ASP 
coutnerparts can so easily hot-patch. We also use JSP precompilation to improve 
performance so it's not so easy to patch JSPs either.

Anyways! Cheers

PS: this was not a rant ... just inquisitive ;)

I hope hot deployment and redeployment is a reality. However, there are issues
when the webapp tries to interact with some services which reside in the system
classloader (logging here). Packaging webapps a little differently could solve
the problems for now.

Allistair, I'll be glad to continue this discussion on the mailing list and try 
and explain why I think reloading an app in-place has only limited usage in 
production environments.  This (Bugzilla) is not the right forum for 
discussions.

I'm closing this item as it's not a Tomcat bug, and a link to it has been added 
to the Tomcat FAQ.

I ran into this issue, and having written a trivial test case, I can confirm 
that this zilla has nothing to do with logging (commons or log4j) or parent 
classloaders etc.  No amount of fiddling with library classloading will fix it.

The real cause is that Tomcat is invalidating (through a call to stop()) the 
WebappClassLoader belonging to a servlet instance whose service() method runs a 
little longer than usual in some conditions (>2 seconds is enough).  This can 
happen under heavy (server, DB) load, with threads still running in service(), 
and may occur in any lifecycle event that runs StandardWrapper.unload() e.g. 
stop, undeploy, reload, shutdown.  If the longish service() method needs to 
load a class after unload() loses its patience, WebappClassLoader throws a 
ThreadDeath.

Might want to leave this one open until I get a chance to log a couple of more 
concise zillas.

Please do not reopen the report. If you look in the code, you'll see that if you
issue a stop or reload, it will wait only a limited amount of time for current
requests to complete.

Sorry, the FAQ points to this issue but it still isn't clear to me how to fix it.

Tomcat/bin contains commons-logging-api.jar
Each webapp ships with its own log4j*.jar files

so what should I be changing in my configuration to prevent a ThreadDeath?
Please clarify.

*** Bug 36250 has been marked as a duplicate of this bug. ***

*** Bug 36250 has been marked as a duplicate of this bug. ***

*** Bug 36250 has been marked as a duplicate of this bug. ***

*** Bug 36250 has been marked as a duplicate of this bug. ***

Tomcat developers, do you still insist that this bug is invalid even though it
has  4 duplicates?

(In reply to comment #35)
> Tomcat developers, do you still insist that this bug is invalid even though it
> has  4 duplicates?

Still in whine mode ? Obviously, duplicates never make something invalid,
especially since the 4 duplicates were filed by the same person. Duh.

I have another small tweak which removes the exception in some cases (trading
this for a stack trace, possible CL failure, and some memory leaking), but the
fact remains that the setup you are using for logging is broken. Apparently you
don't feel concerned.

i think i know how to reproduce and prevent this error.  assistance from the
tomcat developers could enhance the workaround.

the problem (for me) seems to stem from bad commons-logging interaction.  the
commons-logging that the container uses to log errors from my context ends up
discovering the commons-logging.jar and log4j.jar that I have put in my
WEB-APP/lib.  the container ends up with instances of classes loaded by the
WebAppClassloader associated with the initial instance (initial start cycle) of
the webapp.  when reload happens, that classloader becomes invalid, and any
further classloading that is attempted through it will cause ThreadDeath.

However, any classes previously loaded are (apparently) still OK.  so the main
cause of ThreadDeath can be avoided if all the classes that need to be loaded
are loaded while the initial WebAppClassloader is still valid (i.e. before the
first reload).  

the easiest way to force all the classes to load is to LOG A MESSAGE from the
appropriate context.  until a message has been logged, not all classes in the
logging 'execution path' will have been accessed, and so there will be some
class loading that happens during the first logged message.  after that, that
execution path will be 'loaded' and can run till the cows come home.

for me, the easiest thing to do is force a JSP error. maybe have a JSP called
cant_compile.jsp, and hit it as soon as you start up tomcat.  then you will have
NO problems until you shutdown tomcat again.

an easy solution not requiring any changes at all is to put:

log4j.logger.org.apache.catalina.session=DEBUG

into log4j.properties.  you will not get TOO much debug crapola in your logs
(about 15 lines on startup/shutdown).

here's where the tomcat developers come in.  if tomcat logged something as soon
as startup is complete - AT A SUFFICIENTLY HIGH LOGGING LEVEL (eg info) that it
will be feasible to have logging enabled at that level. 

in particular, changing the debug statement at line 655 of StandardManager.java
to info seems a likely candidate.

then users could set (in log4j.properties or equiv):

log4j.logger.org.apache.catalina.session=INFO

and not get superfluous debugging statements.

(In reply to comment #37)
> i think i know how to reproduce and prevent this error.  assistance from the
> tomcat developers could enhance the workaround.

I would disagree, your suggestion simply masks your own problem.

Logging isn't the only culprit in the world, and your solution only touches (and
loads) those segments of the logging infrastructure it uses at that time for
that message.


What TC could do to help the developer is improve that ThreadDeath message and
help the developer identify the exact thread of execution (by name and thread
id) and the class it was trying to load at the time.  This feels like a step in
the right direction it helps the developers stamp out their bugs and does not
encrouch into an ugly half-baked solution the maintainers rightfully want to
keep out.


I'm still toying with the idea of keeping generation(s) of WebAppClassLoader's
around, this idea does have merit for me especially when you keep generations of
all modifications to the web-app around (between container restarts) there are
other good side effects too like hot JAR replace on Win32 where JAR locking
might be an issue.  But this sort of development process has to be driven from
an IDE.

(In reply to comment #38)
Hi, 

We are getting this issue, any pointers to fix this would greatly help us. I 
don' see any practical solution being offered in any of the responses below. 
Any help would be greatly appreciated. There is no LOG4J in the stack trace!!!

utility.UpdateHelper is our application code.


thanks in advance
boni
Tomcat Version: 5.0.28 (using AXIS 1.1), windows platform

The stack trace is :
java.lang.ThreadDeath
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1229)
	at org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1189)
	at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:88)
	at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:195)
	at javax.xml.parsers.DocumentBuilderFactory.newInstance
(DocumentBuilderFactory.java:98)
	at utility.UpdateHelper.TextValueOfElement(UpdateHelper.java:200)
	at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at com.yasutech.qrules.network.k.if(ExpressionEvaluator.java:1018)
	at com.yasutech.qrules.network.k.a(ExpressionEvaluator.java:446)
	at com.yasutech.qrules.network.k.if(ExpressionEvaluator.java:88)
	at com.yasutech.qrules.network.Activation.evaluateExpression
(Activation.java:964)
	at com.yasutech.qrules.network.Activation.changeVariable
(Activation.java:640)
	at com.yasutech.qrules.network.Activation.performAssignAction
(Activation.java:626)
	at com.yasutech.qrules.network.Activation.decideCourseOfAction
(Activation.java:204)
	at com.yasutech.qrules.network.Activation.fire(Activation.java:160)
	at com.yasutech.qrules.network.r.try(Agenda.java:94)
	at com.yasutech.qrules.network.Rete.run(Rete.java:1330)
	at com.yasutech.qrules.rete.ReteAdapter.fireRules(ReteAdapter.java:112)
	at com.yasutech.qrules.rete.ReteRuleEngineAPIHelper.evaluateFacts
(ReteRuleEngineAPIHelper.java:620)
	at com.yasutech.qrules.rete.ReteRuleEngineAPIHelper.executeRuleset
(ReteRuleEngineAPIHelper.java:333)
	at com.yasutech.qrules.rete.ReteEngine.invokeRuleset
(ReteEngine.java:964)
	at webservice.example1.server.MyRuleEngineService.invokeRuleset
(MyRuleEngineService.java:366)
	at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.axis.providers.java.RPCProvider.invokeMethod
(RPCProvider.java:402)
	at org.apache.axis.providers.java.RPCProvider.processMessage
(RPCProvider.java:309)
	at org.apache.axis.providers.java.JavaProvider.invoke
(JavaProvider.java:333)
	at org.apache.axis.strategies.InvocationStrategy.visit
(InvocationStrategy.java:71)
	at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
	at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
	at org.apache.axis.handlers.soap.SOAPService.invoke
(SOAPService.java:481)
	at org.apache.axis.server.AxisServer.invoke(AxisServer.java:323)
	at org.apache.axis.transport.http.AxisServlet.doPost
(AxisServlet.java:854)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
	at org.apache.axis.transport.http.AxisServletBase.service
(AxisServletBase.java:339)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
(ApplicationFilterChain.java:237)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter
(ApplicationFilterChain.java:157)
	at org.apache.catalina.core.StandardWrapperValve.invoke
(StandardWrapperValve.java:214)
	at org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:104)
	at org.apache.catalina.core.StandardPipeline.invoke
(StandardPipeline.java:520)
	at org.apache.catalina.core.StandardContextValve.invokeInternal
(StandardContextValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke
(StandardContextValve.java:152)
	at org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:104)
	at org.apache.catalina.core.StandardPipeline.invoke
(StandardPipeline.java:520)
	at org.apache.catalina.core.StandardHostValve.invoke
(StandardHostValve.java:137)
	at org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:104)
	at org.apache.catalina.valves.ErrorReportValve.invoke
(ErrorReportValve.java:118)
	at org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:102)
	at org.apache.catalina.core.StandardPipeline.invoke
(StandardPipeline.java:520)
	at org.apache.catalina.core.StandardEngineValve.invoke
(StandardEngineValve.java:109)
	at org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:104)
	at org.apache.catalina.core.StandardPipeline.invoke
(StandardPipeline.java:520)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
	at org.apache.coyote.tomcat5.CoyoteAdapter.service
(CoyoteAdapter.java:160)
	at org.apache.coyote.http11.Http11Processor.process
(Http11Processor.java:799)
	at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnectio
n(Http11Protocol.java:705)
	at org.apache.tomcat.util.net.TcpWorkerThread.runIt
(PoolTcpEndpoint.java:577)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
(ThreadPool.java:683)
	at java.lang.Thread.run(Thread.java:595)
> (In reply to comment #37)
> > i think i know how to reproduce and prevent this error.  assistance from the
> > tomcat developers could enhance the workaround.
> I would disagree, your suggestion simply masks your own problem.
> Logging isn't the only culprit in the world, and your solution only touches 
(and
> loads) those segments of the logging infrastructure it uses at that time for
> that message.
> What TC could do to help the developer is improve that ThreadDeath message and
> help the developer identify the exact thread of execution (by name and thread
> id) and the class it was trying to load at the time.  This feels like a step 
in
> the right direction it helps the developers stamp out their bugs and does not
> encrouch into an ugly half-baked solution the maintainers rightfully want to
> keep out.
> I'm still toying with the idea of keeping generation(s) of WebAppClassLoader's
> around, this idea does have merit for me especially when you keep generations 
of
> all modifications to the web-app around (between container restarts) there are
> other good side effects too like hot JAR replace on Win32 where JAR locking
> might be an issue.  But this sort of development process has to be driven from
> an IDE.

(In reply to comment #39)
Some additional information:

This ThreadDeath is just killing the tomcat (java.exe process itself disappears 
from the process list). We have to restart tomcat. Java version being used is 
jdk 1.5.0_04. 

thanks
boni
> (In reply to comment #38)
> Hi, 
> We are getting this issue, any pointers to fix this would greatly help us. I 
> don' see any practical solution being offered in any of the responses below. 
> Any help would be greatly appreciated. There is no LOG4J in the stack trace!!!
> utility.UpdateHelper is our application code.
> thanks in advance
> boni
> Tomcat Version: 5.0.28 (using AXIS 1.1), windows platform
> The stack trace is :
> java.lang.ThreadDeath
> 	at org.apache.catalina.loader.WebappClassLoader.loadClass
> (WebappClassLoader.java:1229)
> 	at org.apache.catalina.loader.WebappClassLoader.loadClass
> (WebappClassLoader.java:1189)
> 	at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:88)
> 	at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:195)
> 	at javax.xml.parsers.DocumentBuilderFactory.newInstance
> (DocumentBuilderFactory.java:98)
> 	at utility.UpdateHelper.TextValueOfElement(UpdateHelper.java:200)
> 	at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:585)
> 	at com.yasutech.qrules.network.k.if(ExpressionEvaluator.java:1018)
> 	at com.yasutech.qrules.network.k.a(ExpressionEvaluator.java:446)
> 	at com.yasutech.qrules.network.k.if(ExpressionEvaluator.java:88)
> 	at com.yasutech.qrules.network.Activation.evaluateExpression
> (Activation.java:964)
> 	at com.yasutech.qrules.network.Activation.changeVariable
> (Activation.java:640)
> 	at com.yasutech.qrules.network.Activation.performAssignAction
> (Activation.java:626)
> 	at com.yasutech.qrules.network.Activation.decideCourseOfAction
> (Activation.java:204)
> 	at com.yasutech.qrules.network.Activation.fire(Activation.java:160)
> 	at com.yasutech.qrules.network.r.try(Agenda.java:94)
> 	at com.yasutech.qrules.network.Rete.run(Rete.java:1330)
> 	at com.yasutech.qrules.rete.ReteAdapter.fireRules(ReteAdapter.java:112)
> 	at com.yasutech.qrules.rete.ReteRuleEngineAPIHelper.evaluateFacts
> (ReteRuleEngineAPIHelper.java:620)
> 	at com.yasutech.qrules.rete.ReteRuleEngineAPIHelper.executeRuleset
> (ReteRuleEngineAPIHelper.java:333)
> 	at com.yasutech.qrules.rete.ReteEngine.invokeRuleset
> (ReteEngine.java:964)
> 	at webservice.example1.server.MyRuleEngineService.invokeRuleset
> (MyRuleEngineService.java:366)
> 	at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:585)
> 	at org.apache.axis.providers.java.RPCProvider.invokeMethod
> (RPCProvider.java:402)
> 	at org.apache.axis.providers.java.RPCProvider.processMessage
> (RPCProvider.java:309)
> 	at org.apache.axis.providers.java.JavaProvider.invoke
> (JavaProvider.java:333)
> 	at org.apache.axis.strategies.InvocationStrategy.visit
> (InvocationStrategy.java:71)
> 	at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
> 	at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
> 	at org.apache.axis.handlers.soap.SOAPService.invoke
> (SOAPService.java:481)
> 	at org.apache.axis.server.AxisServer.invoke(AxisServer.java:323)
> 	at org.apache.axis.transport.http.AxisServlet.doPost
> (AxisServlet.java:854)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
> 	at org.apache.axis.transport.http.AxisServletBase.service
> (AxisServletBase.java:339)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
> (ApplicationFilterChain.java:237)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter
> (ApplicationFilterChain.java:157)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke
> (StandardWrapperValve.java:214)
> 	at org.apache.catalina.core.StandardValveContext.invokeNext
> (StandardValveContext.java:104)
> 	at org.apache.catalina.core.StandardPipeline.invoke
> (StandardPipeline.java:520)
> 	at org.apache.catalina.core.StandardContextValve.invokeInternal
> (StandardContextValve.java:198)
> 	at org.apache.catalina.core.StandardContextValve.invoke
> (StandardContextValve.java:152)
> 	at org.apache.catalina.core.StandardValveContext.invokeNext
> (StandardValveContext.java:104)
> 	at org.apache.catalina.core.StandardPipeline.invoke
> (StandardPipeline.java:520)
> 	at org.apache.catalina.core.StandardHostValve.invoke
> (StandardHostValve.java:137)
> 	at org.apache.catalina.core.StandardValveContext.invokeNext
> (StandardValveContext.java:104)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke
> (ErrorReportValve.java:118)
> 	at org.apache.catalina.core.StandardValveContext.invokeNext
> (StandardValveContext.java:102)
> 	at org.apache.catalina.core.StandardPipeline.invoke
> (StandardPipeline.java:520)
> 	at org.apache.catalina.core.StandardEngineValve.invoke
> (StandardEngineValve.java:109)
> 	at org.apache.catalina.core.StandardValveContext.invokeNext
> (StandardValveContext.java:104)
> 	at org.apache.catalina.core.StandardPipeline.invoke
> (StandardPipeline.java:520)
> 	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
> 	at org.apache.coyote.tomcat5.CoyoteAdapter.service
> (CoyoteAdapter.java:160)
> 	at org.apache.coyote.http11.Http11Processor.process
> (Http11Processor.java:799)
> 	at 
> 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnectio
> n(Http11Protocol.java:705)
> 	at org.apache.tomcat.util.net.TcpWorkerThread.runIt
> (PoolTcpEndpoint.java:577)
> 	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
> (ThreadPool.java:683)
> 	at java.lang.Thread.run(Thread.java:595)
> > (In reply to comment #37)
> > > i think i know how to reproduce and prevent this error.  assistance from 
the
> > > tomcat developers could enhance the workaround.
> > I would disagree, your suggestion simply masks your own problem.
> > Logging isn't the only culprit in the world, and your solution only touches 
> (and
> > loads) those segments of the logging infrastructure it uses at that time for
> > that message.
> > What TC could do to help the developer is improve that ThreadDeath message 
and
> > help the developer identify the exact thread of execution (by name and 
thread
> > id) and the class it was trying to load at the time.  This feels like a 
step 
> in
> > the right direction it helps the developers stamp out their bugs and does 
not
> > encrouch into an ugly half-baked solution the maintainers rightfully want to
> > keep out.
> > I'm still toying with the idea of keeping generation(s) of 
WebAppClassLoader's
> > around, this idea does have merit for me especially when you keep 
generations 
> of
> > all modifications to the web-app around (between container restarts) there 
are
> > other good side effects too like hot JAR replace on Win32 where JAR locking
> > might be an issue.  But this sort of development process has to be driven 
from
> > an IDE.

(In reply to comment #0)
> Many times I've got a java.lang.ThreadDeath exception when trying to reload a
> webapp using the ant task. 
> The call stack of the last time is:
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: init: Associated with
> Deployer 'localhost'
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: init: Global resources 
are
> available
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: restart: Reloading web
> application at '/jn'
> 2004-01-23 12:25:38 StandardContext[/manager]Manager: ManagerServlet.reload
[/jn]
> java.lang.ThreadDeath
>         at
> org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1253)
>         at
> org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1213)
>         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315)
>         at org.apache.log4j.HTMLLayout.format(HTMLLayout.java:129)
>         at org.apache.log4j.net.SMTPAppender.sendBuffer(Unknown Source)
>         at org.apache.log4j.net.SMTPAppender.append(Unknown Source)
>         at org.apache.log4j.AppenderSkeleton.doAppend
(AppenderSkeleton.java:221)
>         at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders
(AppenderAttachableImpl.java:57)
>         at org.apache.log4j.Category.callAppenders(Category.java:187)
>         at org.apache.log4j.Category.forcedLog(Category.java:372)
>         at org.apache.log4j.Category.log(Category.java:864)
>         at org.apache.commons.logging.impl.Log4JLogger.error
(Log4JLogger.java:192)
>         at
> org.apache.catalina.session.StandardManager.start(StandardManager.java:692)
>         at org.apache.catalina.core.StandardContext.start
(StandardContext.java:4216)
>         at
> org.apache.catalina.core.StandardContext.reload(StandardContext.java:2992)
>         at
> org.apache.catalina.manager.ManagerServlet.reload(ManagerServlet.java:1019)
>         at org.apache.catalina.manager.ManagerServlet.doGet
(ManagerServlet.java:377)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
(ApplicationFilterChain.java:284)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter
(ApplicationFilterChain.java:204)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke
(StandardWrapperValve.java:256)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardContextValve.invokeInternal
(StandardContextValve.java:245)
>         at
> org.apache.catalina.core.StandardContextValve.invoke
(StandardContextValve.java:199)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke
(AuthenticatorBase.java:594)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:149)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:195)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:164)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:149)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke
(StandardEngineValve.java:156)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:972)
>         at org.apache.coyote.tomcat5.CoyoteAdapter.service
(CoyoteAdapter.java:211)
>         at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:805)
>         at
> 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnectio
n(Http11Protocol.java:696)
>         at
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:605)
>         at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
(ThreadPool.java:677)
>         at java.lang.Thread.run(Thread.java:536)

(In reply to comment #0)
> Many times I've got a java.lang.ThreadDeath exception when trying to reload a
> webapp using the ant task. 
> The call stack of the last time is:
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: init: Associated with
> Deployer 'localhost'
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: init: Global resources 
are
> available
> 2004-01-23 12:25:37 StandardContext[/manager]Manager: restart: Reloading web
> application at '/jn'
> 2004-01-23 12:25:38 StandardContext[/manager]Manager: ManagerServlet.reload
[/jn]
> java.lang.ThreadDeath
>         at
> org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1253)
>         at
> org.apache.catalina.loader.WebappClassLoader.loadClass
(WebappClassLoader.java:1213)
>         at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315)
>         at org.apache.log4j.HTMLLayout.format(HTMLLayout.java:129)
>         at org.apache.log4j.net.SMTPAppender.sendBuffer(Unknown Source)
>         at org.apache.log4j.net.SMTPAppender.append(Unknown Source)
>         at org.apache.log4j.AppenderSkeleton.doAppend
(AppenderSkeleton.java:221)
>         at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders
(AppenderAttachableImpl.java:57)
>         at org.apache.log4j.Category.callAppenders(Category.java:187)
>         at org.apache.log4j.Category.forcedLog(Category.java:372)
>         at org.apache.log4j.Category.log(Category.java:864)
>         at org.apache.commons.logging.impl.Log4JLogger.error
(Log4JLogger.java:192)
>         at
> org.apache.catalina.session.StandardManager.start(StandardManager.java:692)
>         at org.apache.catalina.core.StandardContext.start
(StandardContext.java:4216)
>         at
> org.apache.catalina.core.StandardContext.reload(StandardContext.java:2992)
>         at
> org.apache.catalina.manager.ManagerServlet.reload(ManagerServlet.java:1019)
>         at org.apache.catalina.manager.ManagerServlet.doGet
(ManagerServlet.java:377)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
(ApplicationFilterChain.java:284)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter
(ApplicationFilterChain.java:204)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke
(StandardWrapperValve.java:256)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardContextValve.invokeInternal
(StandardContextValve.java:245)
>         at
> org.apache.catalina.core.StandardContextValve.invoke
(StandardContextValve.java:199)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke
(AuthenticatorBase.java:594)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:149)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:195)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:164)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:149)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke
(StandardEngineValve.java:156)
>         at
> org.apache.catalina.core.StandardValveContext.invokeNext
(StandardValveContext.java:151)
>         at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>         at org.apache.catalina.core.ContainerBase.invoke
(ContainerBase.java:972)
>         at org.apache.coyote.tomcat5.CoyoteAdapter.service
(CoyoteAdapter.java:211)
>         at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:805)
>         at
> 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnectio
n(Http11Protocol.java:696)
>         at
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:605)
>         at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
(ThreadPool.java:677)
>         at java.lang.Thread.run(Thread.java:536)

For your information, I've tracked down at least one underlying cause of this
kind of failure. During shutdown, Tomcat tries to serialize HTTP sessions to
disk. If a session contains Hibernate objects associated with a hibernate
session which has already been closed, then object.writeReplace() will throw an
exception. This, in turn, causes the serialization process to failure and
Tomcat's reload operation fails as well.

I'd expect Tomcat to be more robust. Specifically, it should emit a warning that
"serialization on shutdown" has failed but should not fail the overall reload
operation. Secondly, if an exception is thrown during serialization it should
print out the stack-trace, not just the exception message (which is what it does
now).

hm .. we see thread death still once in a while and i've noticed a couple 
recently that occur around the time of a stack trace serializable exception. we 
store disconnected hibernate pojos in session objects also.

(In reply to comment #43)
> hm .. we see thread death still once in a while and i've noticed a couple 
> recently that occur around the time of a stack trace serializable exception. we 
> store disconnected hibernate pojos in session objects also.

After studying this thread for a while, I finally reached the previous post and
it opened my eyes. Never place *connected* hibernate pojos in the session or
a ThreadDeath will occur on reload.