Bug 59130 - Definer's way to parse URLs from classloader breaks with recent Java 9 builds (b108)
Summary: Definer's way to parse URLs from classloader breaks with recent Java 9 builds...
Alias: None
Product: Ant
Classification: Unclassified
Component: Core (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal (vote)
Target Milestone: 1.9.7
Assignee: Ant Notifications List
Depends on:
Reported: 2016-03-05 22:23 UTC by Uwe Schindler (ASF)
Modified: 2016-03-06 10:02 UTC (History)
0 users

Patch (1.46 KB, patch)
2016-03-05 22:33 UTC, Uwe Schindler (ASF)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Uwe Schindler (ASF) 2016-03-05 22:23:45 UTC
Copy from the mail to the OpenJDK mailing list. Lucene found this bug while testing Java 9 build 108.

In fact, this is a bug in OpenJDK, but in general the way how the Definer class (base of taskdef, typedef,...) checks if an URL returned by the classloader is an XML file (to parse as Antlib) or a properties file is likely to break on non-standard URLs, like those containing a fragment ("#release" in the Jave 9 build).

Nevertheless we should fix the bug:

We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid error message:

/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support the nested "matches" element.

What happened was that Ant was not loading its internal conditions/tasks/type definitions anymore, so the build system does not know almost any type anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml" from its own JAR file anymore. Instead it printed some strange debugging output (which looked totally broken).

Some debugging showed that Ant tried to parse its own antlib.xml with the conditions and other types as a properties file instead of XML. This also caused the broken logging output (fragments of the XML file printed to log file without any hints).

The reason for this behaviour is the following:

In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returnes URLs like: 


Now in Java 9b108 the following is returned:


And here Ant breaks. Ant checks for the file extension of the string (because it may load definitions from both XML and properties files). So it does endsWith(".xml") and of course this now returns false. The effect is that Ant tries to load its own task definitions as a java properties file instead of XML. Of course this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap as everything to build is missing.

The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org/apache/tools/ant/taskdefs/Definer.java?av=f#259

In fact it converts the URL string and checks with .endsWith('.xml'), which breaks, if an URL fragment is there. A quick workaround is to replace toString() by getPath() which only returns URL path without fragment and without scheme/host (SSP). This is better than the old code, but still has the problem that getPath() may returns the string URL encoded. Ideally one should use url.toURI().getPath(), which decodes, but this is likely to break with jar: URLs. In addition, ".xml" should never be URL-encoded, so URL decoding does not matter. The patch is a one-liner and makes startup of Ant under Java 9 b108 work again. I will send a patch in a minute.

Unfortunately Apache Ivy has a similar problem while loading ivysettings from its JAR file, so the Lucene build is still broken. But anyways, this bug should be fixed to prevent similar problem in the future.
Comment 1 Uwe Schindler (ASF) 2016-03-05 22:33:19 UTC
Created attachment 33629 [details]

Patch fixing the issue
Comment 2 Stefan Bodewig 2016-03-06 10:02:13 UTC
Patch applied.

Thanks a lot for your analysis and for the patch. And in particular thank you for running the Lucene tests against the EA builds and catching bugs for all of us.