This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Imported a large project and while Netbeans was scanning a directory containing a large number of Jar files an OOM error occurred. java.lang.OutOfMemoryError: Java heap space Dumping heap to /xx/.netbeans/6.9.1/var/log/heapdump.hprof ... Heap dump file created [266043785 bytes in 44.703 secs] I have gziped the heapdump down to: 76713150 bytes. Will try to attach to this defect.
Please use this link for attaching such a large file: http://upload.megashare.com/
Heap dump is uploaded here: http://www.megashare.com/2411524 pwd: netbeans
Jardo, please evaluate. For further details please contact thurka.
I have huge troubles to manipulate with the snapshot. For example "compute retained size" did not finish in twelve hours. Otherwise it seems most of the space is taken by WeakHashMap and its entries (about two millions of instances). I am yet trying to figure out who is holding these entries.
Looks like the WeakHashMap is the one from masterfs.NamingFactory. It is supposed to contain more than a million of values, but when looking for FileName subclasses, there is just 14!
Tomáš H. helped me analyse the dump. The stacktraces section shows that there is a local variable of type OutOfMemoryError in RepositoryUpdater. at parsing.impl.indexing.RepositoryUpdater$AbstractRootsWork.scanBinary(:3075) Local Variable: java.lang.OutOfMemoryError#8 Local Variable: java.util.HashMap#16379 Local Variable: java.util.HashMap#16380 This very likely means that the OOME happened while executing one indexBinary method. Then the finalize section of try statement catched the OOME and tried to do some clean up. The cleanup succeeded, but anyway it was too late, the OOME has been reported. The indexBinary seems to collect all FileObject and keep them in memory. This explains while there is are WeakHashMaps with one million of entries, which are mostly null. They have been referenced while indexBinary was on stack, but as soon as the execution jumped to finalize section, they could be cleaned up. There is not much I can do about this in filesystems. The problem is trying to hold too much fileobjects in memory. There should be some limit in the parsing.api or the scanning shall be done incrementally.
The only way how this may happen is that the user has s unarchived binary root with 2M class files.
Fixed jet-main d8452ef7b87a I've fixed the number of active FileObjects. The RU does not collect them but it walks the folder structure as it does before. In my opinion it's useless contract abused by some indexers. If RU does not detect changes it's OK, the indexer gets this info and does nothing. But when RU detects a change it just tells indexer "something has changed" and the indexer is responsible to find out what has changed (it has to walk the folder structure again). I will look at other BinaryIndexers and probably do this incompatible change to improve the performance. Anyway it's not so important as class folder on class path is not very common.
Integrated into 'main-golden', will be available in build *201010280000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/d8452ef7b87a User: Tomas Zezula <tzezula@netbeans.org> Log: #189281:OutOfMemoryError: Two Millions of GCed FileName objects created by parsing Api
I've looked at binary indexers in NB source and it seems that only JavaBinaryIndexer depends on the abused contract of isAllFilesIndexing. So I will remove the contract at all. It will take some time as I need to check performance effects on indexers.
Removed the contract as described above: jet-main 6307aa8ff816
Integrated into 'main-golden', will be available in build *201011020000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/6307aa8ff816 User: Tomas Zezula <tzezula@netbeans.org> Log: #189281:OutOfMemoryError: Two Millions of GCed FileName objects created by parsing Api