Bug 52899 - JMXProxy Servlet MBeanDumper exposing non-properties causing exceptions/memory leak
Summary: JMXProxy Servlet MBeanDumper exposing non-properties causing exceptions/memor...
Status: RESOLVED INVALID
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Manager (show other bugs)
Version: 7.0.26
Hardware: PC Windows Vista
: P2 minor (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-03-13 15:00 UTC by Jonathan Hodges
Modified: 2012-03-16 07:59 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Hodges 2012-03-13 15:00:57 UTC
Once we upgraded to 7.0.26 we see the following exception in the logs when viewing the Tomcat JDBC pool properties through the JMXProxy Servlet.  

As you can see in the following the data is retrieved correctly but a SEVERE exception appears in the logs.




http://localhost:8080/manager/jmxproxy/?qry=bean:name=eventingDataSource


OK - Number of results: 1 

Name: bean:name=eventingDataSource 
modelerType: org.apache.tomcat.jdbc.pool.DataSource 
MaxIdle: 100 
JdbcInterceptors: 
org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer 
DefaultTransactionIsolation: -1 
ValidationQuery: SELECT 1 FROM dual 
TestOnConnect: false 
Password: Password not available as DataSource/JMX operation. 
AbandonWhenPercentageFull: 0 
TestOnReturn: false 
Username: xxxx 
NumActive: 0 
MinIdle: 10 
PoolSize: 4 
ValidationInterval: 1800000 
Connection: 
ProxyConnection[PooledConnection[oracle.jdbc.driver.T4CConnection@17f8b39]] 
NumIdle: 2 
AccessToUnderlyingConnectionAllowed: true 
Url: jdbc:oracle:thin:@(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = 
localhost)(PORT = 1580))(CONNECT_DATA =(SERVER = DEDICATED)( SERVICE_NAME = 
anywhere.com )(INSTANCE_NAME = xxx)(SID = xxx))) 
Size: 4 
Active: 2 
AlternateUsernameAllowed: false 
TestOnBorrow: true 
LoginTimeout: 6 
MaxActive: 100 
MaxAge: 0 
SuspectTimeout: 0 
Pool: org.apache.tomcat.jdbc.pool.ConnectionPool@9283bf 
NumTestsPerEvictionRun: 0 
Name: Tomcat Connection Pool[1-6300663] 
MaxWait: 6000 
PoolProperties: ConnectionPool[defaultAutoCommit=null; defaultReadOnly=null; 
defaultTransactionIsolation=-1; defaultCatalog=null; 
driverClassName=oracle.jdbc.driver.OracleDriver; maxActive=100; maxIdle=100; 
minIdle=10; initialSize=10; maxWait=6000; testOnBorrow=true; 
testOnReturn=false; timeBetweenEvictionRunsMillis=30000; 
numTestsPerEvictionRun=0; minEvictableIdleTimeMillis=30000; 
testWhileIdle=false; testOnConnect=false; password=xxxxx; 
url=jdbc:oracle:thin:@(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = 
localhost)(PORT = 1580))(CONNECT_DATA =(SERVER = DEDICATED)( SERVICE_NAME = 
anywhere.com )(INSTANCE_NAME = xxx)(SID = xxx))); username=xxx; 
validationQuery=SELECT 1 FROM dual; validationInterval=1800000; 
accessToUnderlyingConnectionAllowed=true; removeAbandoned=true; 
removeAbandonedTimeout=60; logAbandoned=true; connectionProperties=null; 
initSQL=null; 
jdbcInterceptors=org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer; 
jmxEnabled=true; fairQueue=false; useEquals=false; abandonWhenPercentageFull=0; 
maxAge=0; useLock=false; dataSource=null; dataSourceJNDI=null; 
alternateUsernameAllowed=false; 
PoolName: Tomcat Connection Pool[1-6300663] 
WaitCount: 0 
JmxEnabled: true 
TestWhileIdle: false 
UseEquals: false 
UseLock: false 
DriverClassName: oracle.jdbc.driver.OracleDriver 
RemoveAbandonedTimeout: 60 
DbProperties: {user=xxxx, password=xxxx} 
Idle: 2 
LogAbandoned: true 
FairQueue: false 
PoolSweeperEnabled: true 
ConnectionAsync: 
org.apache.tomcat.jdbc.pool.ConnectionPool$ConnectionFuture@e9c2d3 
JdbcInterceptorsAsArray: 
Array[org.apache.tomcat.jdbc.pool.PoolProperties$InterceptorDefinition] of 
length 2 

org.apache.tomcat.jdbc.pool.PoolProperties$InterceptorDefinition@125f027 

org.apache.tomcat.jdbc.pool.PoolProperties$InterceptorDefinition@1b99628 
RemoveAbandoned: true 
TimeBetweenEvictionRunsMillis: 30000 
MinEvictableIdleTimeMillis: 30000 
InitialSize: 10 



Mar 6, 2012 9:18:20 AM org.apache.catalina.mbeans.MBeanDumper dumpBeans 
SEVERE: Error getting attribute bean:name=eventingDataSource PooledConnection 
javax.management.MBeanException: RuntimeException thrown in RequiredModelMBean 
while trying to invoke operation getPooledConnection 
        at 
javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1091) 
        at 
javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955) 
        at 
org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:90) 
        at 
javax.management.modelmbean.RequiredModelMBean.getAttribute(RequiredModelMBean.java:1358) 
        at 
org.springframework.jmx.export.SpringModelMBean.getAttribute(SpringModelMBean.java:109) 
        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) 
        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) 
        at 
org.apache.catalina.mbeans.MBeanDumper.dumpBeans(MBeanDumper.java:82) 
        at 
org.apache.catalina.manager.JMXProxyServlet.listBeans(JMXProxyServlet.java:172) 
        at 
org.apache.catalina.manager.JMXProxyServlet.doGet(JMXProxyServlet.java:121) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) 
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) 
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at 
org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108) 
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224) 
        at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169) 
        at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581) 
        at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) 
        at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) 
        at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) 
        at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) 
        at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) 
        at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987) 
        at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579) 
        at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
        at java.lang.Thread.run(Thread.java:619) 
Caused by: java.lang.ClassCastException: $Proxy5 cannot be cast to 
org.apache.tomcat.jdbc.pool.PooledConnection 
        at 
org.apache.tomcat.jdbc.pool.DataSourceProxy.getPooledConnection(DataSourceProxy.java:156) 
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
        at java.lang.reflect.Method.invoke(Method.java:597) 
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:37) 
        at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) 
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
        at java.lang.reflect.Method.invoke(Method.java:597) 
        at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:244) 
        at 
javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074) 
        ... 30 more 
Mar 6, 2012 9:18:22 AM org.apache.catalina.mbeans.MBeanDumper dumpBeans 
SEVERE: Error getting attribute bean:name=eventingDataSource XAConnection 
javax.management.MBeanException: Exception thrown in RequiredModelMBean while 
trying to invoke operation getXAConnection 
        at 
javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1101) 
        at 
javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:955) 
        at 
org.springframework.jmx.export.SpringModelMBean.invoke(SpringModelMBean.java:90) 
        at 
javax.management.modelmbean.RequiredModelMBean.getAttribute(RequiredModelMBean.java:1358) 
        at 
org.springframework.jmx.export.SpringModelMBean.getAttribute(SpringModelMBean.java:109) 
        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) 
        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) 
        at 
org.apache.catalina.mbeans.MBeanDumper.dumpBeans(MBeanDumper.java:82) 
        at 
org.apache.catalina.manager.JMXProxyServlet.listBeans(JMXProxyServlet.java:172) 
        at 
org.apache.catalina.manager.JMXProxyServlet.doGet(JMXProxyServlet.java:121) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:621) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) 
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) 
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at 
org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108) 
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224) 
        at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169) 
        at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581) 
        at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) 
        at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) 
        at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) 
        at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) 
        at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) 
        at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987) 
        at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579) 
        at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
        at java.lang.Thread.run(Thread.java:619) 
Caused by: java.sql.SQLException: Connection from pool does not implement 
javax.sql.XAConnection 
        at 
org.apache.tomcat.jdbc.pool.DataSourceProxy.getXAConnection(DataSourceProxy.java:134) 
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
        at java.lang.reflect.Method.invoke(Method.java:597) 
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:37) 
        at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) 
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
        at java.lang.reflect.Method.invoke(Method.java:597) 
        at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:244) 
        at 
javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1074) 
        ... 30 more 




Apparently MBeanDumper is exposing any getter even if it isn't a property i.e. getPooledConnection.  Since this throws an exception trying to access getPooledConnection the connection is never closed and returned to the pool properly leading to abandoned connections causing a memory leak.

You can see more details in the user thread - http://tomcat.10.n6.nabble.com/What-is-the-best-way-to-view-Tomcat-JDBC-pool-properties-in-Tomcat-7-td4557182.html
Comment 1 Mark Thomas 2012-03-14 15:20:24 UTC
The MBeanDumper only accesses the attributes that are there. It is not responsible for creating the MBeans in the first place.

Tomcat isn't creating this MBean so Tomcat is not responsible for incorrectly exposing pooledConnection as an attribute with a getter rather than an operation. You can tell this by looking at the MBean name. Tomcat uses a distinctive naming convention for MBeans [1] and the name used below does not match it.

[1] http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/java/org/apache/catalina/core/NamingContextListener.java?view=annotate (line 743 onwards)
Comment 2 Jonathan Hodges 2012-03-16 02:31:11 UTC
Why does the exact same config not throw an exception in the JMXProxy in Tomcat 6.0.26?



(In reply to comment #1)
> The MBeanDumper only accesses the attributes that are there. It is not
> responsible for creating the MBeans in the first place.
> 
> Tomcat isn't creating this MBean so Tomcat is not responsible for incorrectly
> exposing pooledConnection as an attribute with a getter rather than an
> operation. You can tell this by looking at the MBean name. Tomcat uses a
> distinctive naming convention for MBeans [1] and the name used below does not
> match it.
> 
> [1]
> http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/java/org/apache/catalina/core/NamingContextListener.java?view=annotate
> (line 743 onwards)
Comment 3 Mark Thomas 2012-03-16 07:59:53 UTC
Because in that version, all exceptions are silently swallowed.