Bug 52853

Summary: @HandlesTypes not resolving classes when added as repository directory
Product: Tomcat 7 Reporter: Benjamin Muschko <benjamin.muschko>
Component: CatalinaAssignee: Tomcat Developers Mailing List <dev>
Severity: normal CC: cbeams
Priority: P2    
Version: 7.0.26   
Target Milestone: ---   
Hardware: Macintosh   
OS: All   
Attachments: Patch to reproduce Bug 52853

Description Benjamin Muschko 2012-03-08 02:55:40 UTC
I am running an embedded container that loads the web application classes and external JARS using the method WebappLoader#addRepository(java.lang.String). One of the external JARs is spring-web with version 3.1. That JAR file provides the class SpringServletContainerInitializer annotated with @HandlesTypes to load implementations of WebApplicationInitializer for web apps that don't provide a web.xml.

In my application I provide an implementation of WebApplicationInitializer. This is found if I add my compiled class files as JAR file using WebappLoader#addRepository(java.lang.String) - the web application gets configured correctly. However, adding the classes directory instead will not resolve the class. This sounds like some kind of class loading issue to me.

As a side-note the web application is loaded just fine when you provide a web.xml (Servlet compatibility < 3.0). All classes can be resolved and used. You can have a look at the source code here: https://github.com/bmuschko/gradle-tomcat-plugin/blob/master/plugin/src/main/groovy/org/gradle/api/plugins/tomcat/TomcatRun.groovy#L70
Comment 1 Mark Thomas 2012-03-09 19:24:13 UTC
Supporting locations for JARs outside of WEB-INF/lib and classes outside of WEB-INF/classes goes beyond what the specification requires but since they are regularly requested features Tomcat does have mechanisms for doing this.

However, for performance reasons, not all of the functionality that is enabled for the standard WEB-INF locations works for the extended locations. In this case, JARs are scanned by default, directories are not. To change this you need to explicitly configure the JarScanner [1] with scanAllDirectories="true".

The users list is the place to seek further assistance if you require it.

[1] http://tomcat.apache.org/tomcat-7.0-doc/config/jar-scanner.html
Comment 2 Benjamin Muschko 2012-03-10 17:10:31 UTC
Thanks for the pointer. Unfortunately, setting scanAllDirectories to true didn't work for me. I will take this to the users list.
Comment 3 Benjamin Muschko 2012-03-14 23:31:53 UTC
Unfortunately, I didn't get a satisfying answer on the Tomcat user mailing list: http://tomcat.10.n6.nabble.com/HandlesTypes-not-resolving-classes-when-added-as-repository-directory-td4565554.html. Therefore, I am going to reopen this as a bug as there potentially might be an issue. Can you either give me other suggestions or have a deeper look? I'd be happy to provide examples or source code. Thanks!
Comment 4 Mark Thomas 2012-03-15 09:03:44 UTC
The simplest possible test case (as source code) that exhibits this issue would be great. I can't stress the "simplest possible" bit enough. Based on the description, I'd expect the simplest possible example to be a few small classes. Ideally, you should based you test case on the Tomcat unit tests and provide a diff against tomcat/trunk since that way the test can become part of the standard test suite. Don't worry too much about what class you put the test in. We'll move it if necessary.
Comment 5 Benjamin Muschko 2012-03-16 01:39:34 UTC
I will try to provide an example that runs an embedded container. I had a look at this repository https://github.com/apache/tomcat70 but couldn't find an example for running tests on an embedded Tomcat container. Is there an already existing setup that I can retrofit to start up an embedded container (including added the embedded Tomcat JARs to the classpath)?
Comment 6 Konstantin Kolinko 2012-03-16 03:08:06 UTC
(In reply to comment #5)
> I will try to provide an example that runs an embedded container.

That would be a question for the users@ list, but I'll answer shortly. See
1) http://tomcat.apache.org/svn.html
2) RUNNING.txt
3) TomcatBaseTest class

You may also want to look at bug #52669 and r1244718 that fixed it. It might be that something is still missing there.
Comment 7 Konstantin Kolinko 2012-03-16 03:09:03 UTC
> 2) RUNNING.txt
and BUILDING.txt
Comment 8 Mark Thomas 2012-03-27 18:13:35 UTC
Waiting for a test case that reproduces this.
Comment 9 Benjamin Muschko 2012-04-18 12:50:10 UTC
I started to create a test case for this issue but I am not sure if it is actually working as I was not able to run the tests using Ant. When asking the question on the mailing list I didn't get a response: http://tomcat.10.n6.nabble.com/Build-Tomcat-from-sources-on-GitHub-repo-td4884681.html. Do you guys know what I am doing wrong? You can find my test case class (org.apache.catalina.loader.TestHandleTypesWebappLoader.java) here:


Any help would be highly appreciated.
Comment 10 Konstantin Kolinko 2012-04-18 21:53:37 UTC
(In reply to comment #9)
> When asking the
> question on the mailing list I didn't get a response:
> http://tomcat.10.n6.nabble.com/Build-Tomcat-from-sources-on-GitHub-repo-td4884681.html.

There is no such question in the mailing list archives. You were fooled by Nabble  - it did not sent the e-mail on your behalf, or it sent it so that it was rejected. You may notice that that page has a yellow warning line.

I'd say that you have to subscribe to the mailing list properly.

Anyway, you had to search the archives, as you are not the first one asking this question:
Comment 11 Chris Beams 2012-04-19 12:11:17 UTC
Thanks Benjamin for tackling this.  I just ran into it today while putting together a Gradle-based demo showing off Spring 3.1 features (which includes web.xml-free configuration via Spring's WebApplicationInitializer).

Note that the 2.0-SNAPSHOT version of the tomcat7-maven-plugin does work just fine, so it must be possible against the existing embedded Tomcat bits.  Take a look at https://github.com/rstoyanchev/spring-mvc-async-sample for a very simple example of this in action.

Perhaps just studying what the Maven plugin is doing differently and following suit in the Gradle plugin will be enough?
Comment 12 Benjamin Muschko 2012-04-20 02:00:47 UTC
Thanks for the pointer, Chris. I will certainly check out the plugin code.
Comment 13 Benjamin Muschko 2012-04-28 22:40:50 UTC
I cleaned up my test cases and made them work. Steps to reproduce:

1) git clone git://github.com/bmuschko/tomcat70.git
2) Add modules directory to sources.
3) Add test entry to build.properties: test.entry=org.apache.catalina.loader.TestHandleTypesWebappLoader
4) Run "ant test"

You should see the following results:

a) Test case 1 finds SpringServletContainerInitializer as the implementation was added as JAR file.

[junit] INFO: Spring WebApplicationInitializers detected on classpath: [org.apache.tomcat.spring.example.SpringExampleWebApplicationInitializer@3e68cd79]

b) Test case 2 doesn't find SpringServletContainerInitializer as the implementation was added as directory.

[junit] INFO: No Spring WebApplicationInitializer types detected on classpath

I hope this is enough information for you guys to have a look into this issue. Maybe I just don't have the correct setup or am missing a configuration setting. Please let me know if you need more information.
Comment 14 Benjamin Muschko 2012-04-28 22:54:44 UTC
One more comment to my test code on Git. As a step between 1 and 2 you will need to switch to the branch "webapp-3.0-spring". I forgot to mention that.

1.5) git checkout webapp-3.0-spring
Comment 15 Konstantin Kolinko 2012-04-29 00:42:06 UTC
(In reply to comment #14)
> 1.5) git checkout webapp-3.0-spring

If you want this to be included in Tomcat test suite, please make a patch in Unified Diff format and attach to the issue.

That is to cover "intentionally submitted for inclusion in the Work" clause in Apache License.
Comment 16 Benjamin Muschko 2012-04-29 14:31:32 UTC
Created attachment 28694 [details]
Patch to reproduce Bug 52853

As there is a limit on the upload size I could not include the required JAR files. Here's a list of them that you will need to put under the directory lib.

* aopalliance-1.0.jar
* cglib-nodep-2.2.2.jar
* commons-logging-1.1.1.jar
* jcl-over-slf4j-1.6.4.jar
* logback-classic-1.0.0.jar
* logback-core-1.0.0.jar
* slf4j-api-1.6.4.jar
* spring-aop-3.1.1.RELEASE.jar
* spring-asm-3.1.1.RELEASE.jar
* spring-beans-3.1.1.RELEASE.jar
* spring-context-3.1.1.RELEASE.jar
* spring-context-support-3.1.1.RELEASE.jar
* spring-core-3.1.1.RELEASE.jar
* spring-expression-3.1.1.RELEASE.jar
* spring-web-3.1.1.RELEASE.jar
* spring-webmvc-3.1.1.RELEASE.jar

For the first test case you will need to compile the source files and JAR them into a file called app.jar under target/lib. For the second test case you will need to compile the source files to target/classes/main".
Comment 17 Mark Thomas 2012-05-06 21:43:18 UTC
That is really the simplest test case you could come up with? I think it is safe to say it will not be being added to Tomcat with all those dependencies. I'll look at the git based test case shortly.
Comment 18 Benjamin Muschko 2012-05-06 22:03:58 UTC
Well, my use case is that fact that I am using Spring's implementation of WebApplicationInitializer. As you might know Spring does define transitive dependencies that are required to make it work so I could not dumb down the test case.

I am totally fine with it not being added to the Tomcat test suite. First and foremost I want to find out if it is a potential bug in embedded Tomcat or if I am missing some configuration that needs to be set.
Comment 19 Mark Thomas 2012-05-06 22:25:11 UTC
You could have simplified this to just the raw elements without any of the Spring libraries. Hey ho.

I have tracked down what is going on. There are two parts to this.

First of all this is never going to work unless (as per my remarks in comment #1) you enabled directory scanning. Your testStartInternalForClassesDirectory() method is missing the following line to do this:
((StandardJarScanner) context.getJarScanner()).setScanAllDirectories(true);

Second, there is a subtlety to the Jar scanner that isn't clear in the docs and needs to be made clearer. A directory is only scanned (even if scanAllDirectories==true) if it is an expanded JAR file. Tomcat determines this by looking for a META-INF directory. If that doesn't exist, it isn't scanned. Therefore, you also need to create a directory "/target/classes/main/META_INF"

With those two changes, you'll see the behaviour you expect. The bug here is in the documentation that does not make that crystal clear. I'll get the docs updated.
Comment 20 Mark Thomas 2012-05-07 13:59:46 UTC
Docs from trunk (r1335026) and 7.0.x (r1335027) updated and will be included in 7.0.28 onwards.
Comment 21 Benjamin Muschko 2012-05-08 01:46:32 UTC
Thanks for pointing me towards the right solution, Mark. Works!