| Summary: | Slow classpath building with referenced paths. | ||
|---|---|---|---|
| Product: | Ant | Reporter: | Constantine Plotnikov <constantine.plotnikov> |
| Component: | Core | Assignee: | Ant Notifications List <notifications> |
| Status: | NEW --- | ||
| Severity: | enhancement | ||
| Priority: | P2 | ||
| Version: | 1.7.1 | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Attachments: |
The archive with sources and build scripts that reproduce the bug.
The archive with sources and build scripts that reproduce the bug with additional layer of projects. |
||
Slow but correct behavior is undesirable but I'm not sure if it qualifies as a bug. Also, it looks entirely like your problem is not the use of one ref, but the fact that in your "bug" files your classpath is composed of sixteen other paths. Finally, your examples seem to suffer from their own smallness: the time it takes me to run either (against svn trunk) is 12 seconds. Try to run in situation when sources are already compiled to see the difference. The problem is with "run" target. Compilation uses inlined classpath. In the real complex project we got 15 minutes. So there is likely non-polynomial dependency on the size of classpath. 16 paths are actually very small amount (especially considering that they are one element). > small amount (especially considering that they are one element).
Ops. I meant "one new element each".
Created attachment 22494 [details]
The archive with sources and build scripts that reproduce the bug with additional layer of projects.
I have added an additional layer of modules (e1-e2) to the project and created new build scripts. Now run target for ref case takes 30 seconds on my computer (vs 0 for inline case).
svn revision 808421 adds a cache attribute on <path>. If I enable it on <path id="antbug.runtime.module.classpath" cache="true"> my build time goes from 32 seconds to 13 seconds using your example. If I set it on all paths with "runtime" in their name it goes down to 7 seconds. This is still worse than the 2 seconds I get without refs but I don't really see how to improve the situation any further since most of the time is spent making sure we don't have a cycle in the graph of connected references. Given the improvement now, I'm going to lower the importance, but not closing the issue since others may find more tweaks we could apply. |
Created attachment 22487 [details] The archive with sources and build scripts that reproduce the bug. When generating ants scripts, we have discovered that if classpath for java application references other paths using <path refid="<path name>"/> a significant slow down happens for big projects with complex dependency structure for launched java applications and tests. We had 15 minutes for classpath building during execution of test tasks. The stack probed from profiler looks like the following: java.lang.Class.isAssignableFrom(Class) org.apache.tools.ant.types.DataType.getCheckedRef(Class, String, Project) org.apache.tools.ant.types.DataType.getCheckedRef(Project) org.apache.tools.ant.types.DataType.getCheckedRef() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.Path.isFilesystemOnly() org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.isFilesystemOnly() org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) org.apache.tools.ant.types.Path.iterator() org.apache.tools.ant.types.Path.iterator() org.apache.tools.ant.types.resources.Union.getCollection(boolean) org.apache.tools.ant.types.resources.Union.list() org.apache.tools.ant.types.Path.list() org.apache.tools.ant.types.Path.addExisting(Path, boolean) org.apache.tools.ant.types.Path.addExisting(Path) org.apache.tools.ant.types.Path.concatSpecialPath(String, Path) org.apache.tools.ant.types.Path.concatSystemClasspath(String) org.apache.tools.ant.types.CommandlineJava.haveClasspath() org.apache.tools.ant.types.CommandlineJava.addCommandsToList(ListIterator) org.apache.tools.ant.types.CommandlineJava.getCommandline() org.testng.TestNGAntTask.executeAsForked(CommandlineJava, ExecuteWatchdog) org.testng.TestNGAntTask.execute() The method org.apache.tools.ant.types.Path.assertFilesystemOnly(ResourceCollection) looks like the primary CPU eater due to recursive invocation. The attached file contains project that allows to reproduce the problem. There are two ant scripts: The script build-inlined.xml demonstrates situation with inlined classpaths. The run task is executed very fast (ant reports 0 seconds). The script build-ref.xml demonstrates situation with referenced classpaths. The run task is executed for about (3 seconds) and the slowdown is due to classpath handling in ant application itself. The only difference between these tasks is the way classpaths for java task are specified in the "run" target.