Bug 55977 - Jdbc pool keepalive flooding
Summary: Jdbc pool keepalive flooding
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-01-08 10:20 UTC by Luca
Modified: 2014-01-09 17:43 UTC (History)
1 user (show)



Attachments
Jmeter log of the scenario. (15.02 KB, text/plain)
2014-01-08 10:20 UTC, Luca
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luca 2014-01-08 10:20:31 UTC
Created attachment 31181 [details]
Jmeter log of the scenario.

In jdbc load test Keepalive age is not respected with bundled excalibur libraries. Versions tried: 2.9, 2.11, trunk.

For example, consider this configuration:
- keepalive: true
- Max Connection age: 5000
- Validation query: select 1 from dual

Insert in a thread group a jdbc sampler and a timer to execute 1 sample per second.

Enabled debug in jmeter properties:
log_level.jmeter.protocol.jdbc=DEBUG

In the output in jmeter.log, after "Max connection age" seconds it seems that  the flag which marks the connection idle is not reset, and for every following sample a keepalive query is executed:
2014/01/08 10:56:20 DEBUG - jmeter.protocol.jdbc.sampler.JDBCSampler: sampling jdbc
2014/01/08 10:56:20 DEBUG - jmeter.protocol.jdbc.config.DataSourceElement: Got a com.sun.proxy.$Proxy0 from the pool.
2014/01/08 10:56:20 DEBUG - jmeter.protocol.jdbc.config.DataSourceElement: Pinging database after 5999ms of inactivity.
2014/01/08 10:56:20 DEBUG - jmeter.protocol.jdbc.AbstractJDBCTestElement: executing jdbc
2014/01/08 10:56:20 DEBUG - jmeter.protocol.jdbc.config.DataSourceElement: Put a com.sun.proxy.$Proxy0 back into the pool.
2014/01/08 10:56:21 DEBUG - jmeter.protocol.jdbc.sampler.JDBCSampler: sampling jdbc
2014/01/08 10:56:21 DEBUG - jmeter.protocol.jdbc.config.DataSourceElement: Got a com.sun.proxy.$Proxy0 from the pool.
2014/01/08 10:56:21 DEBUG - jmeter.protocol.jdbc.config.DataSourceElement: Pinging database after 6999ms of inactivity.
2014/01/08 10:56:21 DEBUG - jmeter.protocol.jdbc.AbstractJDBCTestElement: executing jdbc

In attach the whole log.

Under heavy load the Oracle DB is flooded by keepalive queries that cause unexpected cpu utilization and latencies (even with dual table). 

I tried the latest jars from excalibur (datasource and pool) and the problem seem to me that is solved:
- excalibur-datasource-1.2.0.jar
- excalibur-pool-api-2.1.jar
- excalibur-pool-impl-2.1.jar
- excalibur-pool-instrumented-2.1.jar

Hope this helps to solve the problem, thanks.

Luca
Comment 1 Sebb 2014-01-08 14:11:29 UTC
(In reply to Luca from comment #0)
> ... 
> Under heavy load the Oracle DB is flooded by keepalive queries that cause
> unexpected cpu utilization and latencies (even with dual table). 
> 
> I tried the latest jars from excalibur (datasource and pool) and the problem
> seem to me that is solved:
> - excalibur-datasource-1.2.0.jar

Did you try datasource 2.1 ?

> - excalibur-pool-api-2.1.jar

I don't think you need the api jar.

> - excalibur-pool-impl-2.1.jar
> - excalibur-pool-instrumented-2.1.jar

We currently use the following:

excalibur-datasource.version = 1.1.1
excalibur-instrument.version = 1.0
excalibur-logger.version    = 1.1
excalibur-pool.version      = 1.2

Which of these did you keep?

> Hope this helps to solve the problem, thanks.

Thanks very much, useful to know.
 
> Luca
Comment 2 Luca 2014-01-08 14:33:22 UTC
> Did you try datasource 2.1 ?

I tried it now and seems working.

> 
> > - excalibur-pool-api-2.1.jar
> 
> I don't think you need the api jar.

Without the jar I get this exception:

2014/01/08 15:22:16 ERROR - jmeter.JMeter: Uncaught exception:  java.lang.NoClassDefFoundError: org/apache/avalon/excalibur/pool/ObjectFactory
        at org.apache.jmeter.protocol.jdbc.config.DataSourceElement.initPool(DataSourceElement.java:162)
[...]
 
> We currently use the following:
> 
> excalibur-datasource.version = 1.1.1
> excalibur-instrument.version = 1.0
> excalibur-logger.version    = 1.1
> excalibur-pool.version      = 1.2
> 
> Which of these did you keep?

Here is the list of excalibur jars that I'm using:

excalibur-instrument-1.0.jar
excalibur-logger-1.1.jar
excalibur-datasource-2.1.jar
excalibur-pool-api-2.1.jar
excalibur-pool-impl-2.1.jar
excalibur-pool-instrumented-2.1.jar


Thanks,
Luca
Comment 3 Sebb 2014-01-08 14:57:53 UTC
(In reply to Luca from comment #2)
> > Did you try datasource 2.1 ?
> 
> I tried it now and seems working.

Great

> > 
> > > - excalibur-pool-api-2.1.jar
> > 
> > I don't think you need the api jar.
> 
> Without the jar I get this exception:
> 
> 2014/01/08 15:22:16 ERROR - jmeter.JMeter: Uncaught exception: 
> java.lang.NoClassDefFoundError:
> org/apache/avalon/excalibur/pool/ObjectFactory
>         at
> org.apache.jmeter.protocol.jdbc.config.DataSourceElement.
> initPool(DataSourceElement.java:162)
> [...]

My bad, I assumed the API contained empty classes.
  
> > We currently use the following:
> > 
> > excalibur-datasource.version = 1.1.1
> > excalibur-instrument.version = 1.0
> > excalibur-logger.version    = 1.1
> > excalibur-pool.version      = 1.2
> > 
> > Which of these did you keep?
> 
> Here is the list of excalibur jars that I'm using:
> 
> excalibur-instrument-1.0.jar
> excalibur-logger-1.1.jar
> excalibur-datasource-2.1.jar
> excalibur-pool-api-2.1.jar
> excalibur-pool-impl-2.1.jar
> excalibur-pool-instrumented-2.1.jar
> 
> Thanks,

Thanks again!
Comment 4 Philippe Mouawad 2014-01-08 21:32:28 UTC
It might be the opportunity to switch to Tomcat pool .
Comment 5 Sebb 2014-01-08 21:59:20 UTC
(In reply to Philippe Mouawad from comment #4)
> It might be the opportunity to switch to Tomcat pool .

Please create a new thread on the dev list about this.
Comment 6 Sebb 2014-01-09 17:43:24 UTC
Fixed:

URL: http://svn.apache.org/r1556884
Log:
Jdbc pool keepalive flooding - update Excalibur versions
Bugzilla Id: 55977

Modified:
    jmeter/trunk/build.properties
    jmeter/trunk/build.xml
    jmeter/trunk/eclipse.classpath
    jmeter/trunk/lib/   (props changed)
    jmeter/trunk/res/maven/ApacheJMeter_parent.pom


URL: http://svn.apache.org/r1556889
Log:
Jdbc pool keepalive flooding
Bugzilla Id: 55977

Modified:
    jmeter/trunk/xdocs/changes.xml