This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | Parsing performance impact | ||
---|---|---|---|
Product: | cnd | Reporter: | artisan <artisan> |
Component: | Code Model | Assignee: | Alexander Simon <alexvsimon> |
Status: | RESOLVED FIXED | ||
Severity: | normal | ||
Priority: | P2 | ||
Version: | 7.0.1 | ||
Hardware: | PC | ||
OS: | Linux | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: | log file |
Description
artisan
2011-10-03 09:27:40 UTC
Could you, please, send you ${userdir}/var/log/message.log file? One recommendation could be: try to use 32-bit JVM and give IDE more memory to do that you can #netbeans -J-d32 -J-Xmx2G For big projects increasing memory limit can show huge difference (i.e. 10 times faster parse) It is a memory consumption problem indeed. From the moment it starts scanning the individual files the memory consumption skyrockets. Looking with VisualVM to what Netbeans is doing. :-) even with 2GB of heap space the garbage collector is very busy. Do you want me to send a heap dump ? Created attachment 111413 [details]
log file
2GB of heap just made it through. :-) However, the memory consumption remains quite high. Even after extensive GC the heap topped off at 1.2 GB. And I'm editing nothing. Just started Netbeans and let it run through its scanning process. 22K of files is really big project. As I see you use 64bit JVM could you try 32-bit (which uses almost twice less memory) I can have a try with that yes. Didn't thought about the reference size for 64bits indeed. Thanks ! In the meantime, another thing that might interest you: Removed a lot of source folders to keep the memory consumption down. But it does only when restarting Netbeans. Removing source folders doesn't lower the memory consumption. (In reply to comment #1) > Could you, please, send you ${userdir}/var/log/message.log file? > > One recommendation could be: try to use 32-bit JVM and give IDE more memory > to do that you can > #netbeans -J-d32 -J-Xmx2G > > For big projects increasing memory limit can show huge difference (i.e. 10 > times faster parse) Can we integrate this knowledge into the product? It seems much more user friendly if once we have information on project size, parsing speed, memory used and memory available (as well as bitness of JVM) we provide user with a meaningful advice. Does not look too difficult to implement, does it? That's a good idea indeed - no other IDE does this by my knowledge. Netbeans' profiler already does something similar; to indicate the burden of the profiling on the system. Last, but not least, it broke something in the indexing of the open projects. Some classes can not be found by the "Open Type" (CTRL-O) dialog. Removing the user directory and restarting did the trick. Although it's probably overkill to delete it all. You might want to reassign to Alexander (In reply to comment #9) > Removing the user directory and restarting did the trick. Although it's > probably overkill to delete it all. You don't have to delete whole userdir. it's enough to delete userdir/var/cache/cnd/model Other way is: in project's context menu select Code Assistance->Reparse Project Thanks ! I just didn't know these options. Btw, after switch to 32bit jvm and increasing memory limit, what is responsiveness of IDE? How long does it take to parse your sources? One more note: normally once parsed we do not reparse project next time you open IDE, we only check for changed files and reparse them and files dependent on them. Alexander, please, implement notification based on number of files, parse time (i.e. how long does it take to go 50% -> 60%) and memory used at checkpoints. fixing, change set: http://hg.netbeans.org/cnd-main/rev/6447c245b999 - show low memory warning in parsing progress message Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/6447c245b999 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - show low memory warning in parsing progress message Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/6447c245b999 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - show low memory warning in parsing progress message fixed, change set: http://hg.netbeans.org/cnd-main/rev/a60c66a0cbe5 IDE shows a warning in the right bottom corner of the main frame if code model parser needs more memory. On click on the warning IDE shows dialog with an explanation of the problem and with a way how to fix problem. Dialog is sown once for IDE session. Implemented following strategy to detect low memory: - IDE consume more then 90% of all available heap. - estimated time of remaining parsing is more then 1 minute. I think information about bitness should be checked as well and in case of 64bit JVM we can propose to run (install) 32bit version (In reply to comment #19) > Implemented following strategy to detect low memory: > - IDE consume more then 90% of all available heap. > - estimated time of remaining parsing is more then 1 minute. Isn't 1 minute a too aggressive threshold? additional fix, change set: http://hg.netbeans.org/cnd-main/rev/c33c6ebac12a - add advice about using 32-bit JVM remove debug trace: http://hg.netbeans.org/cnd-main/rev/eaf469a4eb62 Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/a60c66a0cbe5 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - show dialog if code model parser needs more memory Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/c33c6ebac12a User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - add advice about using 32-bit JVM |