This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 130702 [details] Thread dump In the "Copy Class" dialog, the Usages progress bar stays at 50% for more than 20 minutes. I copied just a single class in the projects window. Regardless of how slow the computer is otherwise, surely this kind of thing must not happen. I lost the log file due to overflow when I tested a file search in comparison but there was nothing in it that we don't already know - WARNING [org.openide.filesystems.FileUtil]: FileUtil.normalizeFile etc.. This happens perhaps every 3 weeks. This time I took a few threads dumps which I think are interesting. Below I am looking only at threads doing file I/O simultaneously because file I/O has been what NetBeans engineers have been hinting at in the past. "Explorer Actions" "Subversion - file status refresh" "org.netbeans.modules.glassfish.common.LogViewMgr$LoggerRunnable - java.io.FileInputStream@1e2e0f0" "org.netbeans.modules.glassfish.common.LogViewMgr$LoggerRunnable - java.io.BufferedInputStream@186efb3" "Editor Parsing Loop (201301230001)" "VersioningAnnotator.annotationRefresh" (2 instances) "AWT-EventQueue-0" 8 runnable threads doing I/O simultaneously. How do we expect this to be performing adequately? On the other hand, the IDE grabs so much CPU that it is almost impossible to do anything in other programs. Because what the IDE does is what it has done before in the same session multiple timea, and the file caches deliver the data instantly, leading to 100% CPU with no gaps whatsoever. So it might be of little value to talk about I/O because all of it translates into CPU load. There was enough memory and no swapping. I have run wih far more of IDE memory consumption without any such problems. Please do not blame my computer. I hope this is an opportunity to pinpoint, or to get closer to pinpointing what causes this unfortunate constellation of threads.
Created attachment 130703 [details] Thread dump
I am not able to reproduce this and the thread dumps don't help much, so lets try something different. Because there are problems with slow io on this machine it would be great to see what exactly is going on on the native level. Please provide following information from the some one run - 1. Self sampling snapshot npss 2. Output from http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx 3. Messages.log Hopefully we will be able to gain some insight into what is going on.
Thanks for your helpful reply. I would still be interested to know why the IDE has 8 active threads all doing I/O simultaneously. Of course I would not expect anyone to replicate this problem, because even I would not be able to do this on demand. As I wrote, this happens to me perhaps once every 3 weeks, and I cannot predict it. Therefore I thought that thread dumps might be useful. Could you please explain to me how I can create a self sampling snapshot npss without warning, after this happens and the computer is frozen? I thought I was lucky that I could get jvisualvm to even do a thread dump in this situation. I can see a pattern though. At the start of a session , the IDE is quite responsive. After more than 6 hours, the performance degrades rapidly, and these types of issues are more likely to occur. The IDE does have issues with multiple threads performing I/O at the same time, which is confirmed by the thread dump. I will attach all the other information I have. The only performance issue I could think of that I feel could be remotely related (I have not seen freezing with it), is when I copy a large project. The IDE starts scanning while the project is copied and this combination takes forever. This scanning I/O competes with the copying I/O and it slows down the copying dramatically. Obviously, doing both simultaneously at this point in time is counter-productive. In fact the IDE makes a wrong assumption that I want to open the copied project wich is wrong. This combination of these flaws to me shows the general approach of the IDE towards performance and I am not surprised at all when I see the computer get stuck. To look into native I/O of this machine would be interesting. But I have some doubts whether any energy spent here is productive. Perhaps in another context where I can reproduce things. Thanks for the suggestion. I will buy another computer with a completely different architecture but still Windows XP. It will be industry standard corporate type desktop with multiple cores and multiple performance rating compared with my current computer but still not super fast. But what would be the point? Performance issues are mostly felt with slow computers. If you don't have a slow computer then how can you pinpoint multithreaded performance problems? Impossible. Please see bug 206513. It was not my problem but I was the only one who could provide you with the data to fix it.
Created attachment 130910 [details] snapshot
Created attachment 130911 [details] snapshot
Both snapshots show that trivial native methods which should take milliseconds on normal machines take even tens of second on this system. But that doesn't help much because we already know that. (In reply to comment #3) > Thanks for your helpful reply. > > I would still be interested to know why the IDE has 8 active threads all doing > I/O simultaneously. This is caused by the slow filesystem - it takes so long to do the io that the threads doing io "accumulate" as different threads schedule further io > dumps might be useful. Could you please explain to me how I can create a self > sampling snapshot npss without warning, after this happens and the computer is > frozen? I thought I was lucky that I could get jvisualvm to even do a thread > dump in this situation. You can initiate the self sampling anytime via memory (performance) toolbar. > > I can see a pattern though. At the start of a session , the IDE is quite > responsive. After more than 6 hours, the performance degrades rapidly, and > these types of issues are more likely to occur. The IDE does have issues with > multiple threads performing I/O at the same time, which is confirmed by the > thread dump. I will attach all the other information I have. What is the memory usage at the time it happens - if it is high it can be due to some small leak building up and consuming memory thus forcing the system to swap and thus killing the performance. > The only performance issue I could think of that I feel could be remotely > related (I have not seen freezing with it), is when I copy a large project. The > IDE starts scanning while the project is copied and this combination takes > forever. This scanning I/O competes with the copying I/O and it slows down the > copying dramatically. Obviously, doing both simultaneously at this point in > time is counter-productive. what do you mean by copying project? do you externally copy some project which is opened in IDE and this triggers scanning of this project? > To look into native I/O of this machine would be interesting. But I have some > doubts whether any energy spent here is productive. Perhaps in another context > where I can reproduce things. Thanks for the suggestion. > > I will buy another computer with a completely different architecture but still > Windows XP. It will be industry standard corporate type desktop with multiple > cores and multiple performance rating compared with my current computer but > still not super fast. > > But what would be the point? Performance issues are mostly felt with slow > computers. If you don't have a slow computer then how can you pinpoint > multithreaded performance problems? Impossible. It is not the case - some performance problems are ,sadly, felt even on fast computers. In conlusion - if this happens to you again, please send us the requested information (1. Self sampling snapshot npss 2. Output from http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx 3. Messages.log) and possibly even heap dump if the memory consumption seems to be high.
(In reply to comment #6) >You can initiate the self sampling anytime via memory (performance) toolbar. In this case this was not possible because the IDE is stuck in a dialog. >What is the memory usage at the time it happens - if it is high it can be due >to some small leak building up and consuming memory thus forcing the system to >swap and thus killing the performance. This is an important point. Memory usage goes up during the session There are some resource leaks. And these leaks cause these performance problems in a way that sometimes after a day everything takes 5 times as long as at the beginning. But no swapping as far as I can see. I have 3GB of RAM and NetBeans gets allocated 1GB, with no problems of getting all of it from the system. I had to set -J-Xmx1000m which is huge - wth 7.1.2 (I run comparisons with identical imported settings) I don't need a setting and no problems at all. In my case the resource leaks weigh on CPU more than on other systems, so I would be looking for OutOfMemoryException in vain because the system would come to a halt due to CPU starvation before this happens. I guess that garbage collection is very expensive. Memory goes up and down by incredible amounts - hundreds of megabytes at the time. But then suddenly the max goes also up to 600, then stays there for an hour, goes up to 650 ... 800 and so on where things get unbearable. It is like a slow death. Please see bug 225157 which might have hints to these leaks. I don't use plugins. I am only getting this with Java EE web development, GlassFish starting and stopping, debugging, a single database connection with database explorer. Next time I will make a heap dump. >what do you mean by copying project? do you externally copy some project which >is opened in IDE and this triggers scanning of this project? In the Project window on a project node you select "Copy" in the context menu. Unlike with VCS checkout, e.g. SubVersion, there is no option to open the project or not after copy. The IDE blindly opens and scans the copied project files WHILE it copies the project. This is not optimal. But I did not do this in this context. My computer cannot be optimised further - it runs at optimum performance. It does not even feel slow. After all, it is the Job of the operating system to optimise resources. I tried a few performance switches in netbeans.conf only to get disastrous results. I stay clear of any changes that can cause complications. I am a professional. In conclusion I would suggest to look for chances to eliminate I/O wherever possible especially simultaneous I/O and perhaps put it into a queue. Even more I would suggest to take a hard look at resource leaks like memory, threads and so on. While these may ultimately manifest themselves as OutOfMemoryException, that OOME would not necessarily be their main impact. Who wants to wait for OOME to happen and what is it worth to find out which thread gets the last chunk of memory? I am getting into the habit of re-starting NetBeans to get rid of the pain. I really don't know what happened since 7.1.2 - 7.1.2 is fast on this computer and it stays fast during the day. I think my computer is great because with it, I can pinpoint and reproduce issues that do happen with other computers but cannot be reproduced on them. It just takes patience, and I am glad that NetBeans engineers have patience, too.
I was reading the symptoms you described and I think that your system is not responsible, because your XP is swapping. It will explain, why I/O is terribly slow. It will explain that you cannot start additional process (VisualVM) to sample NetBeans. You said you have 3G memory, so how much memory was consumed by various programs? 1) Empty XP installation consumes around 250 M 2) every java process consumes size of all java binaries (around 50M) plus size of PermGen plus size of Heap plus size of memory allocated by native C code. In case of NetBeans it is 50+256+1000+60, so NetBeans can consume around 1.3 G 3) I don't know what are your setting for Glassfish, but I guess it will be several hundreds of megabytes. 4) You are probably running also database 5) if we count email client and web browser, we can get over 3G. We can speculate what is causing your problems, but we need some data. Petr asked you for 1. Self sampling snapshot npss 2. Output from http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx 3. Messages.log if you thing there is a memory leak, we need heap dump too. You can also remove 1000M Xmx setting and use default settings for NetBeans. Let us see if it makes any difference.