While processing a large number of files via an 'apt' ant task, apt is failing with an out of memory exception. Altering the task to include a 'memoryMaximumSize' argument does not rectify the situation. The following is emitted regardless of the 'fork=' setting in the ant script: [apt] Compiling 1881 source files [apt] Since fork is false, ignoring memoryMaximumSize setting. [apt] [apt] [apt] The system is out of resources. [apt] Consult the following stack trace for details. [apt] java.lang.OutOfMemoryError: Java heap space I haven't found a way to work around the issue via the script. However, I was able to get the Apt task to do what I need by altering the org.apache.tools.ant.taskdefs.Apt constructor as follows. Original constructor: public Apt() { super(); super.setCompiler(AptExternalCompilerAdapter.class.getName()); setFork(true); } Work-around constructor: public Apt() { super(); super.setCompiler(AptExternalCompilerAdapter.class.getName()); super.setFork(true); } However, the workaround causes the task to emit a different warning: [apt] Since compiler setting isn't classic or modern,ignoring fork setting. [apt] Compiling 1881 source files [apt] Since compiler setting isn't classic or modern,ignoring fork setting. I suspect this implies other changes need to be made to get the apt task to recognize the appropriate memory options.
good catch. we turned forking off because we want people to fork every time (classpath problems) but in fact hadn't fixed the problem because we'd stubbed out the setFork() operator. Expect a fix for Ant1.7.1
fixed in SVN, you can check out and use the new version or stick with your patched edition until ant1.7.1 ships. Thank you for finding a bug that snuck past our testing!
It happens again even with ant 1.7.1 with 64bit JDK 5u16 on Ubuntu Intrepid. I am not able to say if it is specific to ant package on Ubuntu. [local]> ant -version Apache Ant version 1.7.1 compiled on October 3 2008 <target name="compile-jruby" depends="compile-tasks, compile-annotation-binder, check-for-optional-packages"> <!-- Generate binding logic ahead of time --> <apt factory="org.jruby.anno.AnnotationBinder" destdir="${jruby.classes.dir}" debug="true" source="${javac.version}" target="${javac.version}" deprecation="true" encoding="UTF-8"> <classpath refid="build.classpath"/> <classpath path="${jruby.classes.dir}"/> <src path="${src.dir}"/> <patternset refid="java.src.pattern"/> <compilerarg line="-XDignore.symbol.file=true"/> </apt> </target> compile-jruby: [apt] Since compiler setting isn't classic or modern,ignoring fork setting. [apt] Compiling 641 source files to /mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build/classes/jruby [apt] Since compiler setting isn't classic or modern,ignoring fork setting. [apt] warning: Annotation types without processors: [java.lang.Override, java.lang.SuppressWarnings, java.lang.Deprecated, org.jruby.anno.JRubyConstant, org.jruby.anno.JRubyModule, java.lang.annotation.Retention, java.lang.annotation.Target] [apt] Problem encountered during annotation processing; [apt] see stacktrace below for more information. [apt] java.lang.OutOfMemoryError: Java heap space [apt] at java.io.BufferedWriter.<init>(BufferedWriter.java:87) [apt] at java.io.BufferedWriter.<init>(BufferedWriter.java:70) [apt] at java.io.PrintStream.init(PrintStream.java:83) [apt] at java.io.PrintStream.<init>(PrintStream.java:100) [apt] at java.io.PrintStream.<init>(PrintStream.java:62) [apt] at org.jruby.anno.AnnotationBinder$AnnotationBindingProcessor$RubyClassVisitor.visitClassDeclaration(AnnotationBinder.java:90) [apt] at com.sun.tools.apt.mirror.declaration.ClassDeclarationImpl.accept(ClassDeclarationImpl.java:95) [apt] at com.sun.mirror.util.DeclarationScanner.visitClassDeclaration(DeclarationScanner.java:110) [apt] at com.sun.tools.apt.mirror.declaration.ClassDeclarationImpl.accept(ClassDeclarationImpl.java:95) [apt] at com.sun.mirror.util.DeclarationScanner.visitClassDeclaration(DeclarationScanner.java:125) [apt] at com.sun.tools.apt.mirror.declaration.ClassDeclarationImpl.accept(ClassDeclarationImpl.java:95) [apt] at org.jruby.anno.AnnotationBinder$AnnotationBindingProcessor.process(AnnotationBinder.java:60) [apt] at com.sun.mirror.apt.AnnotationProcessors$CompositeAnnotationProcessor.process(AnnotationProcessors.java:60) [apt] at com.sun.tools.apt.comp.Apt.main(Apt.java:454) [apt] at com.sun.tools.apt.main.JavaCompiler.compile(JavaCompiler.java:448) [apt] at com.sun.tools.apt.main.Main.compile(Main.java:1075) [apt] at com.sun.tools.apt.main.Main.compile(Main.java:938) [apt] at com.sun.tools.apt.Main.processing(Main.java:95) [apt] at com.sun.tools.apt.Main.process(Main.java:43) [apt] at com.sun.tools.apt.Main.main(Main.java:34) [apt] Note: Some input files use unchecked or unsafe operations. [apt] Note: Recompile with -Xlint:unchecked for details. [o.jruby.distro]> java -version java version "1.5.0_16-rev" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-rev-b09) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_16-rev-b09, mixed mode) [o.jruby.distro]> javac -version javac 1.5.0_16-rev
I forgot to say: The same target passes on 32bit JDK.
It probably only fails on a 64 bit VM because Sun 64-bit JVM heap requirements are 1.5X 32 bit due entirely to memory pointer length; it probably works on the JRockit JVM which uses short pointers when the heap is <4GB. What is the ant -verbose run? The "no forking" message implies that Apt is running in Ant's own process, so if you set the heap size there then it should propagate down. That would make the problem go away, but not fix any defect.
I set ANT_OPTS to "-Xmx1024m -Dbuild.compiler.debug=true" but it does not help anyway. It would say this should be enough so there might be another problem specific to 64bit JDK. Verbose output for this target (I build o.jruby.distro module from NetBeans sources.) is ~5MB. If you wish I can send zip file Hopefully relevant part is (let me know if you want anything else): [apt] Since compiler setting isn't classic or modern,ignoring fork setting. [apt] Compiling 641 source files to /mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build/classes/jruby [apt] Using external apt compiler [apt] Since compiler setting isn't classic or modern,ignoring fork setting. [apt] Compilation arguments: [apt] '-deprecation' [apt] '-d' [apt] '/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build/classes/jruby' [apt] '-classpath' [apt] '/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build/classes/jruby:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/asm-3.0.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/asm-analysis-3.0.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/asm-commons-3.0.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/asm-tree-3.0.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/asm-util-3.0.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/backport-util-concurrent.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/bnd-0.0.249.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/bytelist-0.1.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/dynalang-0.3.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/emma.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/emma_ant.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/jarjar-1.0rc7.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/jline-0.9.93.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/jna-posix.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/jna.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/joda-time-1.5.1.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/joni.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/junit.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/jvyamlb-0.2.3.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/retroweaver-2.0.5.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/build_lib/retroweaver-rt-2.0.5.jar:/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/lib/bsf.jar:/usr/share/ant/lib/ant-launcher.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/xercesImpl.jar:/usr/share/ant/lib/ant-javamail.jar:/usr/share/ant/lib/ant-apache-oro.jar:/usr/share/ant/lib/ant-commons-logging.jar:/usr/share/ant/lib/ant-apache-bsf.jar:/usr/share/ant/lib/ant-apache-regexp.jar:/usr/share/ant/lib/ant-nodeps.jar:/usr/share/ant/lib/ant-jsch.jar:/usr/share/ant/lib/ant-jmf.jar:/usr/share/ant/lib/ant-apache-log4j.jar:/usr/share/ant/lib/ant.jar:/usr/share/ant/lib/ant-commons-net.jar:/usr/share/ant/lib/ant-apache-resolver.jar:/usr/share/ant/lib/ant-antlr.jar:/usr/share/ant/lib/ant-apache-bcel.jar:/usr/share/ant/lib/ant-trax.jar:/usr/share/ant/lib/ant-junit.jar:/usr/share/ant/lib/ant-jdepend.jar:/usr/share/ant/lib/ant-swing.jar:/usr/java/jdk1.5.0_16_64bit/lib/tools.jar' [apt] '-sourcepath' [apt] '/mnt/local/mslama/netbeans/hg-nbsrc/core-main/o.jruby.distro/unpatched_source/jruby-1.1.4/src' [apt] '-target' [apt] '1.5' [apt] '-encoding' [apt] 'UTF-8' [apt] '-g' [apt] '-XDignore.symbol.file=true' [apt] '-source' [apt] '1.5' [apt] '-factory' [apt] 'org.jruby.anno.AnnotationBinder' [apt] [apt] The ' characters around the executable and arguments are [apt] not part of the command. [apt] Files to be compiled:
looking at the source, that warning about forking is invalid; its the superclass whining. The stack trace implies this is a new JVM (no ant classes in the trace) so we are forking. That leaves the question of whether the memory options are going down to the process or not. Can you now do an ant -debug run and paste in the [apt] bit of that log?
I used memoryMaximumSize="256m" for apt task and it works now. Yes warning about forking is confusing and should be eliminated. Parent process memory settings are not propagated to forked apt task so maximum memory size must be increased explicitly. But it is documented to it is correct. Thank you for help. I assume ant -debug output is not necessary anymore. If you still need it let me know.
Problem is that task does not fail when OutOfMemoryError happens in forked JVM. Is it correct? It makes diagnostic difficult in case of building big project.