Bug 62674

Summary: org.apache.jasper.JspC no longer compiles
Product: Tomcat 8 Reporter: Paul <paul>
Component: JasperAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED FIXED    
Severity: major CC: mhatta
Priority: P2    
Version: 8.5.33   
Target Milestone: ----   
Hardware: PC   
OS: Linux   
Attachments: output of 'strace ant'
output of 'ant -d'

Description Paul 2018-09-04 18:51:34 UTC
The update from 8.5.32 to 8.5.33 has caused org.apache.jasper.JspC to fail.  Symptom is that it hangs forever.  Ant output for 8.5.33:

     [java] Sep 02, 2018 9:36:19 AM org.apache.jasper.servlet.TldScanner scanJars
     [java] INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
     [java] Sep 02, 2018 9:36:19 AM org.apache.jasper.JspC execute
     [java] INFO: Generation completed with [0] errors in [616] milliseconds

vs 8.5.32:

     [java] Sep 02, 2018 12:27:17 PM org.apache.jasper.servlet.TldScanner scanJars
     [java] INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a
complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve
startup time and JSP compilation time.
    [javac] Compiling 68 source files to .../build/classes
     [xslt] Processing .../src/conf/web.xml to .../build/web.xml
     [xslt] Loading stylesheet .../src/webmerge.xsl

Neither debug flags nor strace illustrate the issue. Symptoms are identical across Ubuntu 16.04, 18.04 and FreeBSD 11.2 hosts running java 1.8.0_181 with plenty of disk and ram.
Comment 1 Mark Thomas 2018-09-04 21:51:36 UTC
jspc works for me with a simple test case.

Please provide the simplest possible JSP (or set of JSPs and/or tags) that fails to compile.
Comment 2 Paul 2018-09-09 03:48:16 UTC
Have not (yet) isolated a specific file among the 120+ jsps, taglibs and fragments but it does seem to be related to either the number of files to be compiled or some element common to a subset of jsps.
Comment 3 Paul 2018-09-11 16:26:50 UTC
Also hangs in 8.5.34.
Comment 4 Christopher Schultz 2018-09-11 21:53:24 UTC
Can you post a thread-dump of JspC when it's "hung"? That might shed some light on the situation.
Comment 5 Mark Thomas 2018-09-12 16:11:49 UTC
Dropping severity back to default. This isn't going to block a release.
Comment 6 Paul 2018-09-12 23:58:57 UTC
Created attachment 36147 [details]
output of 'strace ant'
Comment 7 Paul 2018-09-13 00:17:03 UTC
Created attachment 36148 [details]
output of 'ant -d'
Comment 8 Masayuki Hatta 2018-09-15 18:48:25 UTC
Hi,

The same problem was reported to Debian.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=908884

I could reproduce this hung up with Debian-packaged libtomcat8-java 8.5.33-1 and 8.5.34-1.  Installing libtomcat8-java 8.5.32-2 fixes this.  The original reporter also said that he could reproduce the problem on Ubuntu and FreeBSD, so I guess some bug must be entered in the upstream JSP handling between Tomcat8 8.5.32 and 8.5.33.

Please let me know if I could help anything else.
Comment 9 Masayuki Hatta 2018-09-16 02:09:49 UTC
I'm pretty sure it's involved with the following bugs.

https://bz.apache.org/bugzilla/show_bug.cgi?id=53492

https://bz.apache.org/bugzilla/show_bug.cgi?id=62662

Possibly the fix in #62662 was not enough?
Comment 10 Mark Thomas 2018-09-16 20:20:38 UTC
(In reply to Paul from comment #6)
> Created attachment 36147 [details]
> output of 'strace ant'

That isn't what Chris asked for. Please provide a Java thread dump.
Comment 11 Masayuki Hatta 2018-09-17 00:01:03 UTC
I2P's analysis and workaround for this problem:

https://trac.i2p2.de/ticket/2307#comment:1
Comment 12 Mark Thomas 2018-09-17 09:01:34 UTC
The relevant information is:

<quote>
... with new the multithreaded JspC implementation in Tomcat 8.5.33, invoking it from an ant script as we do (with fork=true), the JVM never exits, presumably because the Tomcat thread pool is not marked as daemon.
</quote>

It should be fairly easy to confirm that this is the issue. What is puzzling is that I'd expect this to fail even with a few - or even one - JSPs but comment #2 suggests that that was not the case.
Comment 13 Mark Thomas 2018-09-17 10:24:25 UTC
Progress. If JspC is called directly using the Java task (rather than using the Ant Jasper task) and fork is set to true for the Java task then the hand will be observed.

That it is necessary to use the Java task rather than the Jasper task looks to be the most likely explanation of the differences between the reported behaviour in comment #2 and comment #11.
Comment 14 Mark Thomas 2018-09-17 12:42:33 UTC
Assuming these reports are all for a single bug (which I think is the case but comment #2 makes me slightly unsure)...

Fixed in:
- trunk for 9.0.13 onwards
- 8.5.x for 8.5.35 onwards
- 7.0.x for 7.0.92 onwards
Comment 15 Paul 2018-10-04 14:29:26 UTC
Behavior still occurs in 8.5.34.