Bug 37084 - JspC from ant fails on JSPs that use custom taglib
Summary: JspC from ant fails on JSPs that use custom taglib
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 5
Classification: Unclassified
Component: Jasper (show other bugs)
Version: 5.5.12
Hardware: PC Windows XP
: P2 normal with 3 votes (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-10-14 00:36 UTC by Tim Schafer
Modified: 2009-07-10 03:30 UTC (History)
0 users



Attachments
patch that should fix the issue, against tc6.0.x (557 bytes, patch)
2009-01-03 07:47 UTC, Konstantin Kolinko
Details | Diff
webapp that reproduces the issue when compiled by JspC (1.81 KB, application/zip)
2009-01-03 23:11 UTC, Konstantin Kolinko
Details
Proposed patch - port of bug 46471 fix (11.53 KB, patch)
2009-06-12 15:23 UTC, Mark Thomas
Details | Diff
Proposed patch - port of bug 46471 fix (updated) (11.71 KB, patch)
2009-06-23 10:45 UTC, Konstantin Kolinko
Details | Diff
Proposed patch - port of bug 46471 fix (updated 2) (11.75 KB, patch)
2009-06-23 11:24 UTC, Konstantin Kolinko
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Schafer 2005-10-14 00:36:35 UTC
The following error doesn't occur when using tomcat 5.5.9

Seems to be related to the use of a custom taglib in a JSP
that is included by another JSP


The JSP and custom taglib work fine when served by tomcat.
But the ant build script fails.


jspc:
  [jasper2] Oct 13, 2005 2:54:18 PM org.apache.jasper.JspC processFile
  [jasper2] SEVERE: ERROR-the file '\WEB-INF\jsp\headerComponent.jsp' generated
the following general exception:
  [jasper2] java.lang.NullPointerException
  [jasper2]     at sun.misc.URLClassPath$3.run(URLClassPath.java:316)
  [jasper2]     at java.security.AccessController.doPrivileged(Native Method)
  [jasper2]     at sun.misc.URLClassPath.getLoader(URLClassPath.java:313)
  [jasper2]     at sun.misc.URLClassPath.getLoader(URLClassPath.java:290)
  [jasper2]     at sun.misc.URLClassPath.getResource(URLClassPath.java:160)
  [jasper2]     at java.net.URLClassLoader$1.run(URLClassLoader.java:192)
  [jasper2]     at java.security.AccessController.doPrivileged(Native Method)
  [jasper2]     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
  [jasper2]     at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:133)
  [jasper2]     at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:65)
  [jasper2]     at
org.apache.jasper.JspCompilationContext.load(JspCompilationContext.java:596)
  [jasper2]     at
org.apache.jasper.servlet.JspServletWrapper.loadTagFile(JspServletWrapper.java:209)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.loadTagFile(TagFileProcessor.java:515)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.access$000(TagFileProcessor.java:48)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor$TagFileLoaderVisitor.visit(TagFileProcessor.java:566)
  [jasper2]     at org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1441)
  [jasper2]     at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
  [jasper2]     at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213)
  [jasper2]     at org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2219)
  [jasper2]     at org.apache.jasper.compiler.Node$Root.accept(Node.java:456)
  [jasper2]     at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.loadTagFiles(TagFileProcessor.java:584)
  [jasper2]     at
org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:169)
  [jasper2]     at org.apache.jasper.compiler.Compiler.compile(Compiler.java:286)
  [jasper2]     at org.apache.jasper.JspC.processFile(JspC.java:953)
  [jasper2]     at org.apache.jasper.JspC.execute(JspC.java:1094)
  [jasper2]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [jasper2]     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  [jasper2]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  [jasper2]     at java.lang.reflect.Method.invoke(Method.java:585)
  [jasper2]     at org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:123)
  [jasper2]     at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
  [jasper2]     at org.apache.tools.ant.Task.perform(Task.java:364)
  [jasper2]     at org.apache.tools.ant.Target.execute(Target.java:341)
  [jasper2]     at org.apache.tools.ant.Target.performTasks(Target.java:369)
  [jasper2]     at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
  [jasper2]     at
org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:37)
  [jasper2]     at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
  [jasper2]     at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
  [jasper2]     at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:277)
  [jasper2]     at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:201)
  [jasper2]     at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
  [jasper2]     at org.apache.tools.ant.Task.perform(Task.java:364)
  [jasper2]     at org.apache.tools.ant.Target.execute(Target.java:341)
  [jasper2]     at org.apache.tools.ant.Target.performTasks(Target.java:369)
  [jasper2]     at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
  [jasper2]     at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
  [jasper2]     at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
  [jasper2]     at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
  [jasper2]     at org.apache.tools.ant.Main.runBuild(Main.java:668)
  [jasper2]     at org.apache.tools.ant.Main.startAnt(Main.java:187)
  [jasper2]     at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)
  [jasper2]     at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)
  [jasper2] java.lang.NullPointerException
  [jasper2]     at sun.misc.URLClassPath$3.run(URLClassPath.java:316)
  [jasper2]     at java.security.AccessController.doPrivileged(Native Method)
  [jasper2]     at sun.misc.URLClassPath.getLoader(URLClassPath.java:313)
  [jasper2]     at sun.misc.URLClassPath.getLoader(URLClassPath.java:290)
  [jasper2]     at sun.misc.URLClassPath.getResource(URLClassPath.java:160)
  [jasper2]     at java.net.URLClassLoader$1.run(URLClassLoader.java:192)
  [jasper2]     at java.security.AccessController.doPrivileged(Native Method)
  [jasper2]     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
  [jasper2]     at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:133)
  [jasper2]     at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:65)
  [jasper2]     at
org.apache.jasper.JspCompilationContext.load(JspCompilationContext.java:596)
  [jasper2]     at
org.apache.jasper.servlet.JspServletWrapper.loadTagFile(JspServletWrapper.java:209)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.loadTagFile(TagFileProcessor.java:515)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.access$000(TagFileProcessor.java:48)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor$TagFileLoaderVisitor.visit(TagFileProcessor.java:566)
  [jasper2]     at org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1441)
  [jasper2]     at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
  [jasper2]     at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213)
  [jasper2]     at org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2219)
  [jasper2]     at org.apache.jasper.compiler.Node$Root.accept(Node.java:456)
  [jasper2]     at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
  [jasper2]     at
org.apache.jasper.compiler.TagFileProcessor.loadTagFiles(TagFileProcessor.java:584)
  [jasper2]     at
org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:169)
  [jasper2]     at org.apache.jasper.compiler.Compiler.compile(Compiler.java:286)
  [jasper2]     at org.apache.jasper.JspC.processFile(JspC.java:953)
  [jasper2]     at org.apache.jasper.JspC.execute(JspC.java:1094)
  [jasper2]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [jasper2]     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  [jasper2]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  [jasper2]     at java.lang.reflect.Method.invoke(Method.java:585)
  [jasper2]     at org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:123)
  [jasper2]     at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
  [jasper2]     at org.apache.tools.ant.Task.perform(Task.java:364)
  [jasper2]     at org.apache.tools.ant.Target.execute(Target.java:341)
  [jasper2]     at org.apache.tools.ant.Target.performTasks(Target.java:369)
  [jasper2]     at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
  [jasper2]     at
org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:37)
  [jasper2]     at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
  [jasper2]     at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
  [jasper2]     at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:277)
  [jasper2]     at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:201)
  [jasper2]     at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
  [jasper2]     at org.apache.tools.ant.Task.perform(Task.java:364)
  [jasper2]     at org.apache.tools.ant.Target.execute(Target.java:341)
  [jasper2]     at org.apache.tools.ant.Target.performTasks(Target.java:369)
  [jasper2]     at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
  [jasper2]     at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
  [jasper2]     at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
  [jasper2]     at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
  [jasper2]     at org.apache.tools.ant.Main.runBuild(Main.java:668)
  [jasper2]     at org.apache.tools.ant.Main.startAnt(Main.java:187)
  [jasper2]     at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)
  [jasper2]     at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)

BUILD FAILED
C:\cvs\kls\build.xml:79: The following error occurred while executing this line:
C:\cvs\kls\war.xml:114: org.apache.jasper.JasperException: Unable to compile
class for JSP
Comment 1 Tim Schafer 2006-01-24 00:03:03 UTC
This regression is still present in 5.5.15
Comment 2 Sami Shalabi 2006-08-03 15:29:21 UTC
Regression is still present in 5.17
Comment 3 Tim Schafer 2006-10-10 14:08:32 UTC
This regression is still present in 5.5.20
Comment 4 Mark Thomas 2006-10-23 15:42:36 UTC
This works for me. Please confirm that your environment is configured correctly
and if you still see the problem please attach an appropriate test case such as
the zipped source tree using the smallest possible set of files to reproduce the
problem.
Comment 5 Tim Schafer 2006-12-11 15:28:40 UTC
I've found that adding  caching="false"  to the jasper2 task will work around
this problem.

Updated jasper2 call:
<jasper2
  validateXml="false"
  caching="false"
  uriroot="${basedir}/${webapp.dir}"
  webXmlFragment="${basedir}/${build.dir}/WEB-INF/generated_web.xml"
  outputDir="${basedir}/${build.src.dir}" />


Should this work without the caching="false"  ?
I'll post an example project if that's still needed.


Thanks to Grégoire Rolland for providing an example of a jasper2 call that works
with tomcat 5.5.20 jspc

His example posted here for posterity:
<jasper2
    trimspaces="true" 
    caching="false" 
    compilersourcevm="1.5" 
    compilertargetvm="1.5" 
    poolingenabled="false"  
    classdebuginfo="false" 
    genstringaschararray="true" 
    validateXml="false" 
    addWebXmlMappings="true" 
    smapdumped="false"
    smapsuppressed="true"
    verbose="99"
    webxml="${root.dir}/${ant.project.name}/${web}/WEB-INF/web.xml"
    uriroot="${root.dir}/${ant.project.name}/${web}"
   
webXmlFragment="${root.dir}/${ant.project.name}/${web}/WEB-INF/generated_web.xml"
    outputDir="${root.dir}/${ant.project.name}/${web}/WEB-INF/src" />
Comment 6 Mark Thomas 2006-12-11 17:22:17 UTC
Yes, please provide a test case. It should be the simplest possible project that
exhibits this issue.
Comment 7 Mark Thomas 2009-01-01 13:00:31 UTC
I have just tested this again and I still can't reproduce the issue you are reporting. If you still see this issue with 5.5.27 then, as previously requested, please provide the simplest possible test case that demonstrates the issue.
Comment 8 Konstantin Kolinko 2009-01-03 07:40:35 UTC
For reference:
The following mail thread has stacktrace for TC 6.0.18 reproducing the issue:
http://marc.info/?t=122585333800002&r=1&w=2
Comment 9 Konstantin Kolinko 2009-01-03 07:47:21 UTC
Created attachment 23074 [details]
patch that should fix the issue, against tc6.0.x
Comment 10 Konstantin Kolinko 2009-01-03 08:31:39 UTC
Though I am yet failing to reproduce the issue in current tc6.0.x, I have elaborated a theory why it occurs.

1. The exception is caused by creating JasperLoader instance with
a null baseUrl and trying to use it to load a class, in JspCompilationContext#getJspLoader().

Usually that does not occur, because JspCompilationContext#createOutputDir() is called first.

2. During compilation the #createOutputDir() call does occur: a) in Compiler#isOutDated() when it checks for the full name of the java source / class file, b) later, when Compiler#generateJava() explicitly calls JspCompilationContext#checkOutputDir().

Thus, the compilation is not the route that leads to the preliminary creation of JasperLoader instance.

3. I think that the issue is caused when JspServletWrapper#getDependants() gets called earlier than the compilation of the class occurs.

The events will occur as following:
1) JspServletWrapper#getDependants() calls JspCompilationContext#load() that results in #getJspLoader() call and creation of JasperLoader instance. 
2) The JasperLoader instance is unusable and this call results in an exception.
Nevertheless the created JasperLoader instance is remembered.
3) JspServletWrapper#getDependants() catches the exception, but silently swallows it.
4) The program continues running but breaks later.

Thus, the question is what can cause the early #getDependants() call on a fresh compilation context. I am not able to reproduce it. Maybe there should be some interrelated tag/jsp files (OP mentioned: "Seems to be related to the use of a custom taglib in a JSP that is included by another JSP", but that is not sufficient). Also, the known workaround for the issue (disabling caching in JspC) should somehow fit into the picture.

Nevertheless, I am proposing a patch to TC 6.0, that I attached above. A side effect is that it mkdirs the output directory, but I think that must be OK.
Comment 11 Mark Thomas 2009-01-03 14:41:14 UTC
I am -1 for any patch that guesses at a solution as there is no way to confirm the validity of the patch or that it fixes the root cause rather than hides symptoms.

As previously requested, a test case is required.
Comment 12 Konstantin Kolinko 2009-01-03 23:11:32 UTC
Created attachment 23078 [details]
webapp that reproduces the issue when compiled by JspC

To reproduce the issue:

The following recipe is for Eclipse IDE project for TC 6.0.

1. In your project folder create the following subdirectory and two subdirectories in it:
 bug37084
   /output
   /source

2. Unzip the war file into bug37084/source

3. Create the following Run configuration:
type: Java Application
Main class: org.apache.jasper.JspC
Arguments / Program arguments:  -uriroot bug37084\source -d bug37084\output

4. Run the application

5. A stacktrace is observed:

04.01.2009 10:06:30 org.apache.jasper.JspC processFile
SEVERE: ERROR-the file '\index.jsp' generated the following general exception:
java.lang.NullPointerException
	at sun.misc.URLClassPath$3.run(URLClassPath.java:323)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:320)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:297)
	at sun.misc.URLClassPath.getResource(URLClassPath.java:167)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:192)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:134)
...

6. Delete the generated files from bug37084/output

----------------------------------

The patch that I proposed above does not help. It just results in java.lang.ClassNotFoundException being thrown instead of the NPE.
Comment 13 Konstantin Kolinko 2009-01-04 16:56:09 UTC
From debugging the reproducer:

It appears that there are several (ca. 3) issues that pile up to produce this effect.

Ignore my guesses about #getDependants() from comment 10. It is not related. The proposed patch, though, may be useful.


1. First issue, the trigger:

When JspC is run with caching enabled (the default), the tag library information, that is instances of TagLibraryInfoImpl class, is cached.

When TagLibraryInfoImpl instance is created, its constructor passes two pieces of information to the JspCompilationContext that was used to create it.

That information is not passed if the instance is reused from the cache.

First is, ctxt.createCompiler().getPageInfo() call followed by pageInfo.addDependant(location[0]).  -> See bug 46470 that it causes.

Second is ctxt.setTagFileJarUrl(path, jarFileUrl); call. It triggers this issue.

When tagA.tagx is compiled (it is that tag inside a jar file) from inside of tagB.tagx (it is the tag in WEB-INF/tags) in the reproducer, in TagFileProcessor#loadTagFile() the following call: ctxt.getTagFileJarUrl(tagFilePath)
for the tag file returns null.

Thus JspServletWrapper instance is created there with a null tagFileJarUrl instead of the URL of the jar file.

(The tagFilePath argument in the ctxt.getTagFileJarUrl(tagFilePath) call is just the path of the tag file. It does not include the name of the library -> name collisions are expected -> bug 46471. It may have other cause, but the concept is similar).

The TagFileProcessor#loadTagFile() method then proceeds and calls JspServletWrapper.loadTagFile().

2. JspServletWrapper.loadTagFile() 

It calls ctxt.compile(); and ctxt.load();

The compile() method calls compiler.isOutDated() and skips compilation if that method returns false.

In this case, because of invalid (null) tagFileJarUrl, the isOutDated() call cannot find the source file and returns "false", because the source is not found. The JspCompilationContext.compile() method silently returns without compiling anything.
Is that by design? I think that it is.


Thus, compilation is skipped and we proceed to ctxt.load(). That method calls getJspLoader(), but because compilation never occurred, the baseUrl field is not initialized (null), and the method creates a misconfigured JasperLoader instance.

An attempt to load a class from that JasperLoader instance results in the NullPointerException that is observed.

Applying the patch, that I proposed earlier (attachment 23074 [details]) will result in ClassNotFoundException been thrown instead, though there might be alternative implementations that will achieve the same result.


Probably that NPE will occur also in some other situations, when isOutDated() returns false due to missing source, but I have not yet tried to reproduce that.
Comment 14 Mark Thomas 2009-06-12 15:23:55 UTC
Created attachment 23807 [details]
Proposed patch - port of bug 46471 fix

It appears that porting the patch for bug 46471 is sufficient to fix this issue. I have attached the patch that ports the fix and will propose it for 5.5.x. I also ran the JSP 2.0 TCK with this in place and the tests passed.
Comment 15 Konstantin Kolinko 2009-06-23 10:45:12 UTC
Created attachment 23860 [details]
Proposed patch - port of bug 46471 fix (updated)

This patch, as well as the previous (attachment #23807 [details]) by Mark, both do fix this issue for me in tc5.5 - tested with the reproducer and with our webapp that experienced this issue.

But the previous patch does not fix issue 46471, because one change in TagFileProcessor class was omitted when backporting. Thus I propose the updated patch.
Comment 16 Konstantin Kolinko 2009-06-23 11:24:18 UTC
Created attachment 23861 [details]
Proposed patch - port of bug 46471 fix (updated 2)

Compared with original tc6.0 fix for 46471 (http://svn.apache.org/viewvc?rev=739530&view=rev) and spotted an error in the backport.
Thus, updated patch.
Comment 17 Mark Thomas 2009-07-10 03:30:20 UTC
This has been fixed in 5.5.x and will be included in 5.5.28 onwards.