This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
On creation of a new project (in my case a Zend Framework project) Netbeans (Java) takes up a large chunk of cpu (90%+ at some times) for hours. There is no feedback in the lower right corner (such as scanning projects, etc) while this is going on. I suspect it's the "scanning project" phase but I cannot be sure. I'm attaching some profiler output.
Created attachment 102578 [details] Profiler output from during high cpu usage
Verified on MBP w/ 10.6.5 Has exact same symptoms as OP states.
Created attachment 103211 [details] Snapshot of high cpu Adding snapshot during high CPU use, with no task running.
Are the sources on the local dist are a network disk is mounted?
The files are indeed on a local disk, not on a network share.
Do you have still problem with this? There were done improvements in the scanning area. According to the snapshot attached by AIML_Engr the biggest time is spend in the Collections.sort method, called from TaskManager. The zburnham's snapshot contains many java reaper process, which is suspicious, but I don't have a doubts what does it means. So the opened projects are PHP? It's not clear from your comments. I'm marking the issue as worksfor me, because I haven't seen such behavior yet and I'm not able to reproduce it. Please reopen if it happens with a new NB 7.0 build (Beta 1 is old, try a development build).
Using a dev build of 7.0 isn't an option for me, as the beta build already gives me enough trouble. When is another beta scheduled? I'm reopening this until such time as another beta is released and I can re-evaluate the problem.
The Beta 2 has been announced today. Could you try it with the new beta? Thanks.
Beta2 is available, could you try it and switch off the experimental hints, if you have them switch on? Closing as works for me, please reopen if it happen again without experimental hints turned on. Thanks
How does one turn off the experimental hints in the beta?
You can turn of in Tools->Options->Editor->Hints tab, where you choose PHP language in the combo box. Let me know, whether it helps.
I'm using beta 2 now, and the experimental hints were turned off by default. I've seen some behavior like what was described, but it may be improved. I'll keep using it and let you know.
Feel free to reopen if the problem appears again. Thanks.
The problem is still present in the overnight dev build, 20110225001. I had two projects open and Java went to 100% CPU and stayed there for at least half an hour. I finally ended up quitting out of the IDE because it became so slow as to be unusable; code completion was taking over a minute.
Could you describe more the projects? How many files and how many PHP, JavaScript, HTML? Another attempt is to try new userdir? Just start the ide from command line with --userdir. See more here: http://blogs.sun.com/netbeansphp/entry/userdir. Last think can you attach new snapshot of a part of scanning? Thanks
There is a snapshot attached to this bug already. If the problem recurs I will grab one. One project consists of a few hundred PHP files and a handful of JS and CSS files. I think the problem with this kind of project is that there is a central directory of code that's then symlinked to by over 100 different directories. Could resolving the symlinks be what's causing the issue?
Resolving symlinks can really take longer time. Do you have switch on task list? Could you try to turn off the tasklist?
How do I shut off task scanning? I found Options->Miscellaneous->Tasks but there doesn't appear to be a way to disable it. Do I need to remove all the patterns?
It should be enough to switch off Tasks window. When the window is not displayed, then the tasks shouldn't be collected.
*** Bug 192184 has been marked as a duplicate of this bug. ***
The snapshot from issue #192184 shows the same problem, sorting in Java Task List. There is issue #191717 that is about the sorting in Java Task List. The issue #191717 is marked as fixed in December, could you please provide new snapshots with Beta 2? In one case the fix of issue #191717 was not sufficient and needs to be reopened and this issue marked as duplicate of this or there is another issue.
When you start the IDE with Task list window opened, how many Errors, TODO and other items are there?
I just tried closing the IDE and then opening it when the tasks window was open. The tasks were showing as 100 in all open projects before I closed it, then when I opened it, I had <no tasks> showing. It does not currently appear to be showing the 100% CPU bug.
Created attachment 106599 [details] The snapshot that shows that sorting errors slow down the IDE during scanning I have created php dummy project that has many errors and TODOs. It caused slow down during scanning and according the snapshot the problem is Task List.
@zburnham: So if you have switch off the Task List, then the problem disappear?
I can't be sure at this point. Once the 100%cpu problem occurs once on a project, it seems to not recur on subsequent openings, like the project is completely indexed at that point. I'll try with another project that may not be 'clean' and report back what I find.
I will try summarize, what I have found. There are two problems: 1) The sorting in Task List that introduce unnecessary complexity 2) The HTML validator generates at leas 1 false error for every php file The first problem should be fixed by Tomas and the validator by Marek
>2) The HTML validator generates at leas 1 false error for every php file I'm addressing the issue right now. It requires changes in the csl.api, resp. in the TLIndexer not to use parserResult.getDiagnostics() but to use results from HintsProvider.
fixed the sorting part - http://hg.netbeans.org/core-main/rev/ee6a2ebc4df4, reassigning back to php
Created attachment 106608 [details] Snapshot during high cpu state The attached snapshot is during the state of high cpu usage after opening a project. When I closed the task window, the CPU usage dropped to 'nominal' levels from 100%+.
I've changed the CSL task list provider semantic a bit so the logic is now not to use parser result errors directly, but if there is a HintsProvider for the language, ask it for the error hints and use these items instead of the parser errors. There's the fallback to the parser errors if there's no hints provider. I would like to kindly ask jlahoda and ppisl to review the change. Thanks. web-main#ccb8c94951bf
Just a note: There's a certain incompatibility of the change though it is not an API incompatibility. Many of the HitsProvider implementations relied on the fact that the given RuleContext is their own and directly casted it to their own type. This is however not true during the "dry run" to the HP from the TLIndexer. I've fixed the PHP, EL and JS implementations, but maybe there are others, maybe not in the build. So if you are aware of any, please let me know, I'll fix them as well.
Note #2: There is a wrong assumption in the TLIndexer that the offsets got from the parser result Error-s are document offset. I believe they are supposed to be the embedded ones. Due to this assumption the tasklist errors to editor navigation is broken. Since the lines handling logic in a bit messy, I didn't like to fix it together with the original problem. Anyone is welcomed to fix...
Integrated into 'main-golden', will be available in build *201103020001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/ee6a2ebc4df4 User: Tomas Stupka <tstupka@netbeans.org> Log: issue #191265 - High CPU usage for hours after launching netbeans with a project open
changeset: 189912:7b7239b0706b date: Wed Mar 02 18:48:11 2011 +0100 summary: #191265 - reverting the previous change from the usage of HintsProvider results for the tasklist back to the parserResult errors. Loading of nb document for the indexed fileobject was necessary for the HP implementation, but doing that seemed to caused some race conditions causing the parsing thread to stuck occasionally. The solution for the problem an introduction of a new SPI in the csl.api module which can be used to filter the unwanted parser errors for the tasklist.
Integrated into 'main-golden', will be available in build *201103030057* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/ccb8c94951bf User: Marek Fukala <mfukala@netbeans.org> Log: #191265 - better filtering of errors for takslist
Integrated into 'main-golden', will be available in build *201103040000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/7b7239b0706b User: Marek Fukala <mfukala@netbeans.org> Log: #191265 - reverting the previous change from the usage of HintsProvider results for the tasklist back to the parserResult errors. Loading of nb document for the indexed fileobject was necessary for the HP implementation, but doing that seemed to caused some race conditions causing the parsing thread to stuck occasionally. The solution for the problem an introduction of a new SPI in the csl.api module which can be used to filter the unwanted parser errors for the tasklist.
There were last week other fixes. I'm marking this issue as fixed. zburnham feel free to reopen this issue, if you still have problem with looong scanning.
I can report that this problem still appears to be present in a less severe form in 7.0 RC1. It only happened for a few minutes, but it was still there.
I spoke too soon. As soon as I opened a file, Netbeans started taking up 100%+ CPU and stayed there, going on ten minutes now. The navigator pane is still in "Please Wait..." state. This RC is unusable, at least on OS X.
Please attach snapshot. Without it I can not do nothing.
Please attach the snapshot from the RC1 ide build taken during the high cpu load. Since there have been more changes to the area recently the previously attached snapshots doesn't reflect the current state of things. Reopen the issue once you do that. Thanks in advance.
Created attachment 107432 [details] Snapshot during high cpu state Snapshot, as requested.
The attached snapshot covers 5 seconds from the indexing loop. I don't see any suspicious. The time is split between html, js, php and TL indexing. Could you provide one more snapshot? Also how many files does your project have? Marek do you see something strange in the snapshot?
My project consists of a central directory of code and directories that contain symlinks back to that central directory. The central directory has 1596 files, and there are 183 directories that symlink to those files. In addition, a different central repository (that isn't symlinked to) has 816 files in it. So effectively: (1596*183) + 1596 + 816 = 294480 files or, alternately: 1596 + 183 files = 1779 files 1596 symlinks * 183 = 292068 symlinks Hope this helps. I'll try to get another snapshot; how long should I let the profiler run for?
The important information for me now is the symlinks. Is there a symlink that point to an upper directory so there is "a cycle" in directory structure?
If I understand the question, no, there should not be any cyclic references in the code base, as they would more than likely been spotted by now. This code has been in production for years.
(In reply to comment #44) > Marek do you see something strange in the snapshot? No, it looks ordinary. I reckon the problem is in the amount of files.
I have overlook the number of the files. Let's say that your project has 290 000 files. The symlinks are threated as files. If one file will be indexed 50 ms, then all files could be indexed after 50 x 290 000 / 1000 / 3600 = 4.02 hours. Switch off the Task List that influence the time. But I don't think that reindexing of such source root will be shorter than few hours. You can try to index it over the night (without the task list window on) and we have to expect that there is no recursive symlink. In such case it will be never ending. You can post a 5 or 10 minutes snapshot (if it will not be too huge), whether there will be something visible. I can not imagine how fast will be the ide after the indexing. The query to the index will be slow anyway. I thing without downgrading number of files, there will not be a reasonable solution.
Just a guess, but couldn't NB be made smart enough to recognize a symlink as pointing at a file it's already indexed, and skip re-indexing that file?
The symlinks are problem in Java. The operation to find out whether it's a symlink can take seconds (mainly on Windows). So for the performance reasons we can not use it. The problem should be solved in JDK 1.7. I'm marking this ant wontfix until the symlinks will be supported.