Bug 52025 - 7.0.21 and newer break the JDK 6 Service provider mechanism for loading JDBC Drivers
7.0.21 and newer break the JDK 6 Service provider mechanism for loading JDBC ...
Status: RESOLVED INVALID
Product: Tomcat 7
Classification: Unclassified
Component: Catalina
7.0.21
PC All
: P2 regression with 1 vote (vote)
: ---
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2011-10-13 21:19 UTC by Rick Bullotta
Modified: 2011-10-17 14:15 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rick Bullotta 2011-10-13 21:19:49 UTC
In Tomcat 6 and Tomcat 7.0.20 and earlier, JDBC drivers in a web app's classpath would automatically be loaded using the JDK's service provider mechanisms.  Starting with Tomcat 7.0.21, this no longer works.  I suspect is has something to do with the classloader that is in effect at the time of this initialization, but not certain. It is difficult to debug, because the actual loading occurs in a Sun class sun.misc.Service, for which source is very hard to find!
Comment 1 Mark Thomas 2011-10-13 21:24:37 UTC
Try setting driverManagerProtection="false" for the JreMemoryLeakPreventionListener in server.xml.

If I recall correctly, the service provider mechanism didn't work across multiple web applications with different drivers anyway.
Comment 2 Konstantin Kolinko 2011-10-13 21:26:09 UTC
(In reply to comment #0)
> would automatically be loaded using the JDK's service provider
> mechanisms.

Can you be more specific? Preferably with a step-by-step instruction on how to reproduce this.
Comment 3 Rick Bullotta 2011-10-13 21:31:16 UTC
Mark, thanks for the quick feedback.  That does indeed seem to return it to the pre-7.0.21 behavior.  That said, I still think we'd have to call it a regression issue, since it changes previous behavior and would break existing apps.

It is an issue with the implementation of the JreMemoryLeakPreventionListener or just a changed behavior?
Comment 4 Rick Bullotta 2011-10-13 21:32:06 UTC
Mark, thanks for the quick feedback.  That does indeed seem to return it to the
pre-7.0.21 behavior.  That said, I still think we'd have to call it a
regression issue, since it changes previous behavior and would break existing
apps.

It is an issue with the implementation of the JreMemoryLeakPreventionListener
or just a changed behavior?

(In reply to comment #1)
> Try setting driverManagerProtection="false" for the
> JreMemoryLeakPreventionListener in server.xml.
> If I recall correctly, the service provider mechanism didn't work across
> multiple web applications with different drivers anyway.
Comment 5 Mark Thomas 2011-10-13 21:50:43 UTC
I'll need to do some testing to see how well this actually worked before the change. I have my suspicions about possible failure modes I need to check.

Depending on what the testing shows, this will / will not be treated as a regression but either way the docs will probably end up being updated.
Comment 6 Rick Bullotta 2011-10-13 22:01:25 UTC
(In reply to comment #5)
> I'll need to do some testing to see how well this actually worked before the
> change. I have my suspicions about possible failure modes I need to check.
> Depending on what the testing shows, this will / will not be treated as a
> regression but either way the docs will probably end up being updated.

I can reproduce fairly easily with the following scenario:

Drop a couple JDBC drivers (any will do, MySQL connector and jTDS are two that we use) in the lib directly of a webapp.  When an attempt is made to get a JDBC connection, in the loadInitialDrivers of the DriverManager class, it will (one time) try to get a list of JDBC drivers using the Service Provider mechanism.  7.0.20 and earlier would load and find three drivers (the two in the webapp and one built into the JVM). 7.0.21 and later do not see the drivers in the webapp.  It uses the getCallerClassLoader() method to pass a classloader into the ServiceProvider stuff that is in sun.misc.Service. I'm guessing that the classloader here is obviously behaving differently - just not sure the mechanics, and it sound like you're very familiar with what's going on.

LMK if you want us to test/try anything else.
Comment 7 Mark Thomas 2011-10-16 20:14:52 UTC
I am closing this as INVALID as the Service Provider mechanism for DriverManager is - as I suspected - broken for servlet container environments. Given that it was a lottery if this ever worked before, that it now consistently doesn't work isn't really a regression.

What I have done is updated the documentation to try and explain what is going on and how to work-around it. Basically, just assume the service provider mechanism isn't there and manually register (and deregister) the Driver.

The change for the updated docs is here:
http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/jndi-datasource-examples-howto.xml?r1=1184919&r2=1184918&pathrev=1184919

You may find this presentation on memory leaks useful / interesting:
http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf
Comment 8 Rick Bullotta 2011-10-17 13:52:48 UTC
Thanks, Mark.  Probably the reason they left the Service provider stuff in sun.misc!!!  We'll just go back to the ever-popular Class.forName, which (almost) always works...

(In reply to comment #7)
> I am closing this as INVALID as the Service Provider mechanism for
> DriverManager is - as I suspected - broken for servlet container environments.
> Given that it was a lottery if this ever worked before, that it now
> consistently doesn't work isn't really a regression.
> 
> What I have done is updated the documentation to try and explain what is going
> on and how to work-around it. Basically, just assume the service provider
> mechanism isn't there and manually register (and deregister) the Driver.
> 
> The change for the updated docs is here:
> http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/jndi-datasource-examples-howto.xml?r1=1184919&r2=1184918&pathrev=1184919
> 
> You may find this presentation on memory leaks useful / interesting:
> http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf
Comment 9 Konstantin Kolinko 2011-10-17 14:03:30 UTC
(In reply to comment #7)
> I am closing this as INVALID as the Service Provider mechanism for
> DriverManager is - as I suspected - broken for servlet container environments.

The scan for service providers happens only once when DriverManager initializes. See DriverManager.getDrivers() / DriverManager.loadInitialDrivers(). (JDK 6u26)

1. My understanding regarding DriverManager.getDrivers() call in JreMemoryLeakPreventionListener is that Service Provider mechanism should work if the class is visible to the JreMemoryLeakPreventionListener.

That is, it should work if the driver jar is in $CATALINA_BASE\lib

It might be nice to mention it in the docs, because it might be useful for users, but I have not tested whether it actually works.


2. I agree with Mark that for webapps it is broken at JRE level and cannot be fixed, thus INVALID. Even if you disable the listener, at most one webapp per JVM would be able to make use of it.
Comment 10 Mark Thomas 2011-10-17 14:15:46 UTC
(In reply to comment #9)
> 1. My understanding regarding DriverManager.getDrivers() call in
> JreMemoryLeakPreventionListener is that Service Provider mechanism should work
> if the class is visible to the JreMemoryLeakPreventionListener.
> 
> That is, it should work if the driver jar is in $CATALINA_BASE\lib
> 
> It might be nice to mention it in the docs, because it might be useful for
> users, but I have not tested whether it actually works.

Yes, it does work. I'll add that to the docs.