This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I have a 2 GB RAM machine and I was able to use "Find Usages" without problems in our large project with NB 6.0. With NB 6.1 Beta, I get OutOfMemory. Attaching log.
Created attachment 57982 [details] log
*** This issue has been marked as a duplicate of 129864 ***
OK, I am pretty sure that this is no duplicate of the other issue with parsing. I got this OOM again in NB 6.1 RC1, but this time during Find. So I guess, it is not parsing and not Find Usages, but the display of the result in the output window (same in Find Usages and Find). I have a heap dump of this situation: 2 small projects and 1 large project open, 38 sources open. Memory is at 116 MB. Then a Find over the large project. When the number of matches is about 100 in about 60 files, I get the OOM, because then about 300 MB is allocated. How can I upload the heap dump? It's 31 MB even when heavily packed with bzip2.
Try to upload the file to some file sharing service like www.zupload.com and leave the link here. Please do not change issue priority if you are not willing to fix the issue.
I raised the priority, because I see this as a showstopper for NB 6.1. Here is the link to the heap dump: http://www.mediafire.com/?fcmsmitxujy zupload did not work, but mediafire does.
Thanks for the file. In the heap dump you sent there are two big char arrays (128MB + 67MB). Unfortunately it is impossible to find who created them since there was some GC and arrays contain only binary data. According to messages.log this should really relate to parsing of java files. Are you able to reproduce this OOM? There is D:\P2plus\Trunk\AppServer\P2Java\com\apag\p2plus\p2core\Tool.java in your messages.log. Could you check the file content?
This messages.log was old. I am attaching a recent one, that is related to the heap dump. Is there anything else I can check, when it happens?
Created attachment 60297 [details] messages.log related to the heap dump
As I wrote before. Is it reproducible? Is D:\P2plus\Trunk\AppServer\P2Java\com\apag\p2plus\p2core\Tool.java or other file you search broken? How did you create the dump file? With -J-XX:+HeapDumpOnOutOfMemoryError?
If you want, I can mail you Tool.java (NDA). I used jmap to get the heap dump.
In case the file content is OK it is not necessary to send. I thought it could be some kind of cross-linked file issue. If it is reproducible a new heap dump using -J-XX:+HeapDumpOnOutOfMemoryError would be helpful. It should prevent GC. Strange is that the issue occurs always when the IDE reads from a stream. It looks like some layer (file content provider) provides incorrect content.
I got it reproducable at the moment with a search in a large project. Here is a new automatic dump with -J-XX:+HeapDumpOnOutOfMemoryError: http://www.mediafire.com/?nznzmsjhy1v This was the stacktrace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) at java.lang.StringBuffer.append(StringBuffer.java:306) at java.io.BufferedReader.readLine(BufferedReader.java:345) at java.io.LineNumberReader.readLine(LineNumberReader.java:182) at org.netbeans.modules.search.BasicSearchCriteria.checkFileContent(BasicSearchCriteria.java:642) at org.netbeans.modules.search.BasicSearchCriteria.matches(BasicSearchCriteria.java:575) at org.netbeans.modules.search.SpecialSearchGroup.processSearchObject(SpecialSearchGroup.java:108) at org.netbeans.modules.search.SpecialSearchGroup.doSearch(SpecialSearchGroup.java:88) at org.openidex.search.SearchGroup.search(SearchGroup.java:178) at org.netbeans.modules.search.SearchTask.run(SearchTask.java:124) at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:561) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:986)
So the problem is not with a large project, but rather with a large binary file inside the project (or generally under the search root) and the fact that utilities/search module tries to load the file at once (as it has no newlines). Try to find that file and check searching without it. Utilities/search module should probably try text search only on text/* files, and certainly don't try to count lines (readLine) on binary files.
There are two issues in the Utilities module related to the described problem: - the algorithm for distinguishing text files from binary files is far from perfect - reading by lines - the problem is when the lines are very long None of the above has changed since NB 6.0 so it cannot be a regression in the Utilities module.
OK, I had large heap dumps in the project main folder. After I deleted them, the OOM during Find is gone. So, maybe this was not the same problem as the initial Find Usages problem after all.
Thanks for the additional information. It seems that the heap dump files generated for fighting with the original bug caused that you discovered another bug in the Utilities module. Because the last comments in this bug report are about the issues in the Utilities module, I will not change the category back to "refactoring" and I will track the bugs in the search feature. If you find a way to reproduce the original issue in Find Usages, please file a separate bug report against module "refactoring". This bug will not be fixed in NB 6.1. It will be fixed in the next release.
Because I do not know of any reliable detection of binary files, I will make a quick fix - not search large files of unknown type (MIME-type "content/unknown").
Fixed. Changeset Id: 39af3e9658b4 (http://hg.netbeans.org/main/rev/39af3e9658b4)
Integrated into 'main-golden', available in build *200808280201* on http://bits.netbeans.org/dev/nightly/ Changeset: http://hg.netbeans.org/main/rev/39af3e9658b4 User: Marian Petras <mpetras@netbeans.org> Log: fixed bug #129557 - "Find may consume a lot of memory when searching large unrecognized files"
*** Issue 145383 has been marked as a duplicate of this issue. ***
Reopening - reproduced in NetBeans IDE 6.5 RC1 (Build 200810171318) http://statistics.netbeans.org/exceptions/detail.do?id=132581
I do not of any other fix than the one described in issue #134558, i.e. search FileObjects instead of DataObjects. FileObjects are lightweight (fast to initialize, do not hold much memory), DataObjects are heavyweight.
Removed blocker 155379 because it's blocking (not prevent DataObject creation) only searching for xml files but regular projects mostly have java files. So since issues 134558 and 155380 were fixed and blocker 155379 was removed (as described above) now search for java files should use less memory and performance should be better.
Build: NetBeans IDE Dev (Build 200903310200) VM: Java HotSpot(TM) 64-Bit Server VM, 11.0-b16, Java(TM) SE Runtime Environment, 1.6.0_11-b03 OS: Linux, 2.6.27.7-9-default, amd64 User Comments: Stacktrace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuffer.append(StringBuffer.java:224) at org.netbeans.core.startup.TopLogging$LgStream.write(TopLogging.java:726) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
Created attachment 79214 [details] stacktrace
Build: NetBeans IDE 6.7 (Build 200906241340) VM: Java HotSpot(TM) Client VM, 14.0-b16, Java(TM) SE Runtime Environment, 1.6.0_14-b08 OS: Linux, 2.6.28-14-generic, i386 User Comments: Stacktrace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuffer.append(StringBuffer.java:224) at org.netbeans.modules.xml.wizard.XMLGeneratorVisitor.printModel(XMLGeneratorVisitor.java:217) at org.netbeans.modules.xml.wizard.XMLGeneratorVisitor.visitChildren(XMLGeneratorVisitor.java:163)
Created attachment 85427 [details] stacktrace
Build: NetBeans IDE 6.7.1 (Build 200907230233) VM: Java HotSpot(TM) Client VM, 14.0-b16, Java(TM) SE Runtime Environment, 1.6.0_14-b08 OS: Linux, 2.6.28-14-generic, i386 User Comments: Stacktrace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuffer.append(StringBuffer.java:224) at org.netbeans.modules.xml.wizard.XMLGeneratorVisitor.printModel(XMLGeneratorVisitor.java:217) at org.netbeans.modules.xml.wizard.XMLGeneratorVisitor.visitChildren(XMLGeneratorVisitor.java:163)
Created attachment 85446 [details] stacktrace
Created attachment 85526 [details] stacktrace