This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 183606 - I/O under lock in masterfs's FolderObj.refreshImpl()
Summary: I/O under lock in masterfs's FolderObj.refreshImpl()
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: All All
: P2 normal (vote)
Assignee: Jaroslav Tulach
URL:
Keywords: PERFORMANCE
: 189917 (view as bug list)
Depends on:
Blocks: 169958
  Show dependency tree
 
Reported: 2010-04-06 21:54 UTC by misterm
Modified: 2011-02-18 20:10 UTC (History)
64 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter: 167576


Attachments
nps snapshot (256.00 KB, application/nps)
2010-04-06 21:54 UTC, misterm
Details

Note You need to log in before you can comment on or make changes to this bug.
Description misterm 2010-04-06 21:54:52 UTC
Build: NetBeans IDE Dev (Build 100403-97deba2209a4)
VM: Java HotSpot(TM) Client VM, 10.0-b19, Java(TM) SE Runtime Environment, 1.6.0_04-b12
OS: Windows XP

User Comments:
GUEST: right clicked on a package and said to create new class. It froze when the window appeared.



Maximum slowness yet reported was 76584 ms, average is 13915
Comment 1 misterm 2010-04-06 21:54:59 UTC
Created attachment 96819 [details]
nps snapshot
Comment 2 Jaroslav Tulach 2010-04-13 04:47:58 UTC
Could the global action just be enabled and compute its state outside of AWT thread later?
Comment 3 Vladimir Voskresensky 2010-08-10 13:45:24 UTC
displaying project context menu takes > 50 sec and it blocks EDT.
I don't think it is about java refactoring...
have a look at
http://statistics.netbeans.org/exceptions/detail.do?id=167576
and most time is spent in 
org.netbeans.modules.masterfs.filebasedfs.fileobjects.FileObjectFactory.getFileObject

daily IDE is unusable

Product Version: NetBeans IDE Dev (Build 100809-bac7e8fbd5d8)
Java: 1.6.0_14; Java HotSpot(TM) Client VM 14.0-b16
System: SunOS version 5.10 running on x86; UTF-8; ru_RU (nb)
Comment 4 Vladimir Voskresensky 2010-08-11 07:28:20 UTC
Jarda, can you review, please, imho, something wrong with FS
Comment 5 Vladimir Voskresensky 2010-08-12 10:29:29 UTC
hint:
in home dir I had broken link => when in command line type
cd ~/
and then press tab => long delay

and of course everything is fast in command line when I'm in folders without broken links

=> it seems that NB FS implementation does something with home dir too often without real need (canonicalPaths?) => most of FO operations are slowed down by broken link in home...
Comment 6 Jaroslav Tulach 2010-08-30 07:11:27 UTC
refreshImpl acquires writeAccess and then queries filesystem for its status (File.isDirectory). If the I/O check is slow for any reason, then other (trivial) operations like masterfs's FolderObj.getFileObject() are blocked.

This observation is based on analyses of http://statistics.netbeans.org/exceptions/exception.do?id=415249
Comment 7 David Strupl 2010-09-08 13:45:48 UTC
Why was this tracked under java/refactoring and assigned to Jarda? If it is YAGL in masterfs it should be on filesystems IMHO. Moving it there.
Comment 8 Jaroslav Tulach 2010-10-05 09:21:36 UTC
There are other I/O operations inside FolderObj.refreshImpl(). There is File.listFiles(), but that is just one. There can potentially be many File.isDirectory() calls. Simplest way to move them out is to subclass File and pre-cache the I/O before entering the Mutex lock.
Comment 9 Vladimir Voskresensky 2010-10-19 09:36:53 UTC
Can we, please, at least notify user about problematic file.

http://statistics.netbeans.org/analytics/exception.do?id=431096
55 sec to show project popup menu :-(((
Comment 10 Vladimir Voskresensky 2010-10-19 09:40:15 UTC
there is CR6992581 in bugster against Studio IDE 
----------
On my system, it's taking 3.5 minutes before the file chooser comes up.
- I supplied --userdir to a local disk.  (The userdir.java did not previously exist).
- The project I want to open is on the local disk.
- The CWD is on the local disk.
Why does the file chooser default to my home directory?  Why does it take so long to come up?
-------------
I think it does not matter if I have any "broken" or "slow" links in my
home directory. The point is that the IDE should NEVER look inside my
home directory unless I explicitely told it to look there. There is 
only one subdirectory inside my home that the IDE can use - it is
${HOME}/.sunstudio
Also, the IDE should NEVER use AWT thread to perform any I/O operations.
------------
Comment 11 Vladimir Voskresensky 2010-10-19 09:43:58 UTC
DirecotryChooserUI must prepare completion list out of EDT, not it's blocked by file.isDirectory check for 50 secs
Comment 12 Petr Nejedly 2010-11-06 19:10:22 UTC
Not a YAGL, actually, as the Mutex is per folder in MFS. So only the accesses to the same folder can block.
This might happen quite frequently with refresh-after-window-activated and slow filesystem (as the refresh traverses all the folders, hitting the slow one earlier or later), but should be less of a problem on systems with native FS listeners.
Comment 13 Petr Nejedly 2010-11-10 11:12:02 UTC
Given the fact that the problem should be less frequent/severe now (with native listeners) and is no crash or data loss, I'm reducing the priority.
Comment 14 Vladimir Voskresensky 2010-11-10 11:57:58 UTC
Petr, 
I don't understand now from new description of this bug "what is the problem and test case".
have a look at comment #10 (user problem) and comment #5 (100% testcase)

I've just checked that on any system (native listener is ON) with NFS home #5 gives 2 minutes of AWT freeze.

Studio is targeting enterprise segment customers who have NFS systems, so this is a real blocker (P1)
Comment 15 Petr Nejedly 2010-11-10 13:05:10 UTC
Well  I wasn't looking at the #10, it is unrelated to this problem.
All the reports under this issue are o.o.fs.FileSystem accesses blocked by another thread doing slow I/O under a folder lock. And these cases are backed by profiler snapshots and in most cases related to contention from automatic refresh.

Your #10, #11 talks about FileChooser. You should then file a separate issue against o.n.swing.dirchooser module, preferably with a profile of a thread dump.
Comment 16 Vladimir Voskresensky 2010-11-10 13:57:51 UTC
I agree that #10, #11 can be filed as separate issue

comment #5 is the way for reproducing #3, which is the *problem* of this issue, right?
Comment 17 Petr Nejedly 2010-11-10 14:46:44 UTC
Yes, #3 is THE problem, but #5 its a pathological example (that is, problem on an individual user side, hardly a P1*, not far from arguing the IDE should not freeze when the user's project.xml sits on a server on Mars surface), while most of the other reports have the contention caused by the automatic refresh on window activation.

And I'm not saying this shouldn't be fixed for 7.0.
I'm just saying it is not a stopper for beta. Agreed?

*) Workaround exists, delete the broken link.
Comment 18 Jaroslav Tulach 2011-02-17 09:26:13 UTC
ergonomics#1f6d92de6dd6
Comment 19 Quality Engineering 2011-02-18 10:32:25 UTC
Integrated into 'main-golden', will be available in build *201102180501* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/1f6d92de6dd6
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #183606: Move I/O during refreshing children outside of mutex.writeAccess
Comment 20 Jaroslav Tulach 2011-02-18 20:10:15 UTC
*** Bug 189917 has been marked as a duplicate of this bug. ***