This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 186345 - Debug Session fails while "Checking for external changes"
Summary: Debug Session fails while "Checking for external changes"
Status: RESOLVED WORKSFORME
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: PC Windows Vista
: P2 normal (vote)
Assignee: Jaroslav Tulach
URL:
Keywords:
Depends on: 183308
Blocks:
  Show dependency tree
 
Reported: 2010-05-19 07:09 UTC by bht
Modified: 2010-06-02 16:22 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
thread dump and log files (87.09 KB, application/octet-stream)
2010-05-19 07:09 UTC, bht
Details
snapshot (58.62 KB, application/octet-stream)
2010-05-25 12:12 UTC, Egor Ushakov
Details
new snapshot (150.11 KB, application/octet-stream)
2010-05-27 10:28 UTC, Egor Ushakov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description bht 2010-05-19 07:09:29 UTC
Created attachment 99179 [details]
thread dump and log files

While debugging a web application, the debugger stopped responding while the status bar animation
"Checking for external changes" never stops.

The variables window displays "Please wait ..."
Finally, the current call is no longer identifiable - Call Stack window also "Please wait ..."

No floppy seeks at this time.

Thread dump taken at this stage.

Have to finish debugger session. Clears things up. Can hear floppy seeks even while not changing focus.

I am not used to this level of unstability in previous releases - I hope this is only temporary.
Comment 1 bht 2010-05-19 07:11:40 UTC
Sorry, is 6.9 beta not 6.8
Comment 2 Martin Entlicher 2010-05-19 08:32:34 UTC
Debugger is stuck, because call EventRequestImpl.enable() did not return. It's waiting on a socket, which it can not write to. It looks like the application you're debugging is frozen that it does not accept debugger data.
Debugger certainly can not continue in this state.

In your log file I see several "Exception occurred in target VM" messages. These exceptions occurred during some evaluations in a tooltip in the program you were debugging. Did debugger managed to display the exceptions? These could cause your application to die or freeze...
Comment 3 bht 2010-05-19 09:03:08 UTC
I noticed that I did not get tooltip data when I expected it. But I did not notice any exceptions at this stage. May be I was overwhelmed. If the log data does not give any useful clues to you then I don't know how to help further. Your exception reporter is very clever and Netbeans stability has benefited from it. So may be you can use this to improve the error reporting further so that next time the bug can be caught?

BTW I noticed that the debugger variables window takes a lot of time to build up lists of objects before the freeze, and in other sessions, too.
Comment 4 Martin Entlicher 2010-05-19 09:32:54 UTC
I just meant to say that the exceptions were thrown by your program that you were debugging. This is what "Exception occurred in target VM" means.

There is a java.lang.NullPointerException at org.netbeans.modules.debugger.jpda.expr.TreeEvaluator.invokeVirtual(TreeEvaluator.java:255)
which is on line:
Throwable ex = new InvocationExceptionTranslated(iex, debugger);
This can not throw NPE itself and thus it looks like it's the translated exeption that was thrown by the debugged program. Unfortunately the stack traces are not full, I do not know why.

If no exceptions were displayed to you it can be that debugged did not manage to translate fully the exception that your program threw.

We'd need a sample application that demonstrate this to know exactly what's going on there. Can you please check if your program threw some exceptions? They seem to come from toString() calls.
Comment 5 Martin Entlicher 2010-05-19 09:34:29 UTC
Also, performance of the Variables window should be significantly improved in the latest 6.9 builds, so please check it there...
Comment 6 Martin Entlicher 2010-05-19 11:35:31 UTC
Also, when the debugger is stuck, is some CPU used?

We need more information about this, so resolving as incomplete for now...
Comment 7 Egor Ushakov 2010-05-25 12:12:15 UTC
Created attachment 99430 [details]
snapshot

I have almost the same behavior on Solaris, I started debug session during "checking for external changes" process. And debug session was starting much much slower. I was starting to think that it would never start. CPU usage was almost zero. "Checking external.." process was almost constantly suspended but sometimes working.
Comment 8 Egor Ushakov 2010-05-25 12:17:25 UTC
after session has finally started any activity in debugger takes ~10 second e.g. stack update. I decided to wait until "checking for external changes" process finish. But after that it started again. My machine does not do anything than NB... CPU usage is still zero.
Comment 9 Egor Ushakov 2010-05-25 12:21:18 UTC
forgot to say NB Dev 20100518
Comment 10 Martin Entlicher 2010-05-25 13:13:04 UTC
Debugger could not start because it needs to submit breakpoint first and need to test if they are applicable or not. It was waiting 20 seconds for ClassPath.findResource(), which is waiting for filesystems...

SourceForBinaryQuery.findSourceRoots(), which takes 10 seconds, also spends most of the time sleeping somewhere in filesystems.

In another thread where toolbar actions are loaded ClassPath.getRoots() takes 21 seconds because of the time that is spent in fileystems.

It's clear that the main problem is the disk access. Some Mercurial commands were run, therefore they could consume some disk bandwidth.

I do not think we can solve this in debugger.

"Refresh-After-WindowActivated" thread consumes over 26 seconds in File.list() method. This certainly adds to the disk usage, though it might not be the only problem.
Comment 11 Martin Entlicher 2010-05-25 13:14:43 UTC
Moving to filesystems to evaluate if disc access can be more efficient...
Comment 12 Egor Ushakov 2010-05-26 13:12:41 UTC
I have one Netbeans run in debug session from another Netbeans, so I switch between them quite often. The main issue in my case I think is that "checking..." process is active all the time! As soon as one finish another starts almost immediately.
Comment 13 Tomas Pavek 2010-05-26 15:16:00 UTC
Wasn't this possibly fixed by recent fixes in FS - e.g. bug 186291 or bug 186427?
Comment 14 Antonin Nebuzelsky 2010-05-27 09:38:10 UTC
Egor, can you please comment on this issue today?

Do you see the problem with the latest trunk build?
Comment 15 Egor Ushakov 2010-05-27 09:41:30 UTC
will try
Comment 16 Egor Ushakov 2010-05-27 10:28:35 UTC
Created attachment 99530 [details]
new snapshot

it does not look any better in Dev (Build 201005270001), new snapshot attached
Comment 17 Egor Ushakov 2010-05-27 10:31:40 UTC
line breakpoint toggling was taking ~10 seconds and IDE was not responsive during that
Comment 18 Jaroslav Tulach 2010-05-27 11:02:15 UTC
Please disable "refresh after window is activated" in Tools/Options/Misc/Files to isolate the problem from the after activation refresh influence.
Comment 19 Egor Ushakov 2010-05-27 11:27:38 UTC
line breakpoint toggle slowness is reproducible with ExternalChecking disabled and looks like bug 184250
Comment 20 Egor Ushakov 2010-05-27 12:55:34 UTC
it looks like many problems here because IDE goes to physical file system too often (and from AWT thread also). Having this if fs is busy with external changes checking (and versioning updates) IDE may easily become less responsive. See bug 182064 and bug 183308.
Comment 21 Martin Entlicher 2010-05-27 13:14:22 UTC
In the latest snapshot (deb2.nps) ClassBasedBreakpoint.isEnabled() takes 50 seconds, because it is waiting for filesystem. ClassBasedBreakpoint.remove() spends 31 seconds in AWT waiting on a lock, which is not good, but the core problem is filesystem which makes the IDE unusable anyway.

I'd track AWT performance of ClassBasedBreakpoint.remove() in a separate issue, since it's just made visible by waiting for FS.
Comment 22 Martin Entlicher 2010-05-27 13:17:31 UTC
I've submitted bug #186808 for that.
Comment 23 Tomas Pavek 2010-05-27 14:26:01 UTC
Those 50s is spent in ClassBasedBreakpoint.remove() -> ... ClassPath.findResource() -> ... FileObjectFactory.getFileObject() - which then spends 31s in calls of File.exists() and 19s in waiting. There's 357 calls of getFileObject().

The 31s indicates probably very slow or overloaded filesystem, the 19s waiting is possibly suspicious (from NB FS point of view). The snapshot is with external checking still enabled. Whether the external checking is responsible for the wait is unclear.

Anyway, the main root cause seems to be the slow filesystem. I'd say it is not a stopper then.
Comment 24 Egor Ushakov 2010-05-27 14:39:44 UTC
What do you mean by slow filesystem? I do not have anything active on my machine except for two Netbeans (one run from another in debug mode). All sources are local. My developer NB even have userdir in memory (in /tmp). If filesystem is overloaded then it is because of Netbeans.
Comment 25 Antonin Nebuzelsky 2010-05-27 14:47:38 UTC
How much memory do you have on your system and what is its utilization (used by Netbeans, by other process, free for system caches,...)? And how many projects do you have open in NetBeans? Expensive and system demanding check for external changes could be caused by no space in RAM for FS caches on OS level and huge number of folders which are scanned for the changes.
Comment 26 Tomas Pavek 2010-05-27 14:58:21 UTC
(In reply to comment #24)
> What do you mean by slow filesystem? I do not have anything active on my
> machine except for two Netbeans (one run from another in debug mode). All
> sources are local. My developer NB even have userdir in memory (in /tmp). If
> filesystem is overloaded then it is because of Netbeans.

By slow filesystem I meant e.g. a mounted remote FS, or having some other (external to NB) I/O intensive operation like hg update running on the machine. Now we know it's not the case.
Comment 27 Egor Ushakov 2010-05-27 16:45:38 UTC
I have ~30 project opened in the IDE (all CND  projects). 4Gb of memory (3.6Gb used) + 4Gb swap (0 used) on OpenSolaris. Netbeans run with the default settings. I've just rebooted the machine. Looks like most memory is used for file cache now. NB works really fast. I do not why it is getting worse and worse over the time. Still I do not believe it is a good idea to rely on OS file cache. I will wait a couple days and see what will happen. You can close the issue for now.
Comment 28 Tomas Pavek 2010-06-02 16:22:56 UTC
Ok, closing this issue for now.

BTW we are not relying on FS caches, but expecting certain speed/response time of the FileSystem that is usually achieved on local drives (because of FS caches). Certainly there is a room for making the IDE not touching I/O so often.