Bug 49905 - In cluster, when using DeltaManager memory leak can occur
Summary: In cluster, when using DeltaManager memory leak can occur
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 6
Classification: Unclassified
Component: Cluster (show other bugs)
Version: 6.0.29
Hardware: PC Windows Server 2003
: P2 normal (vote)
Target Milestone: default
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-09-09 07:22 UTC by Ruslan
Modified: 2010-10-07 07:06 UTC (History)
0 users



Attachments
Patch to remove context classloader from threads in ThreadPoolExecutor in tribes (1.41 KB, application/octet-stream)
2010-09-09 07:22 UTC, Ruslan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ruslan 2010-09-09 07:22:00 UTC
Created attachment 26008 [details]
Patch to remove context classloader from threads in ThreadPoolExecutor in tribes

Tested on apache-tomcat-6.0.29 running under jdk 1.6.0_18.

When DeltaManager is instantiated and assigned in StandardContext.start(),
is it done AFTER StandardContext.bindThreads().

DeltaManager, in turn, during initalization, asks for sessions in other nodes,
and this may result in creating threads in ThreadPoolExecutor in tribes.

These threads created with contextClassLoader set to current
webapplication WebAppClassLoader.

This results in memory leak and error message during redeployment in tomcat log:

09/09/2010 14:46:19 S - - WebappClassLoader.clearReferencesThreads:
The web application [/creditdev] appears to have started a thread
named [pool-1-thread-1] but has failed to stop it. This is very likely
to create a memory leak.

Stacktrace:
       at java.util.concurrent.ThreadPoolExecutor.addThread(Unknown Source)
       at java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(Unknown
Source)
       at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
       at org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor.addToQueue(MessageDispatch15Interceptor.java:67)
       at org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:68)
       at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
       at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:87)
       at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
       at org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:216)
       at org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:175)
       at org.apache.catalina.ha.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java:813)
       at org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions(DeltaManager.java:959)
       at org.apache.catalina.ha.session.DeltaManager.start(DeltaManager.java:930)
       at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
       at org.apache.catalina.core.StandardContext.start(StandardContext.java:4559)
       at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
       at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
       at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546)
       at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:637)
       at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:563)
       at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:498)
       at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
       at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
       at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
       at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
       at org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
       at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
       at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:445)
       at org.apache.catalina.core.StandardService.start(StandardService.java:519)
       at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
       at org.apache.catalina.startup.Catalina.start(Catalina.java:581)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
       at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:243)
       at java.lang.Thread.run(Unknown Source)

Proposed solution - implement java.util.concurrent.ThreadFactory in
MessageDispatch15Interceptor
and pass instance on ThreadPoolExecutor executor creation.
This instance must call setContextClassLoader(null) in newThread()
overriden method.
Comment 1 Mark Thomas 2010-10-01 12:35:02 UTC
This has been fixed in trunk and will be included 7.0.3 onwards.

I have proposed the same fix for 6.0.x.
Comment 2 Mark Thomas 2010-10-07 07:06:56 UTC
This has been fixed in 6.0.x and will be included in 6.0.30 onwards.