This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 182192 - mercurial->commit from IDE force full rescan
Summary: mercurial->commit from IDE force full rescan
Status: RESOLVED FIXED
Alias: None
Product: editor
Classification: Unclassified
Component: Parsing & Indexing (show other bugs)
Version: 6.x
Hardware: Sun Solaris
: P2 normal (vote)
Assignee: Vitezslav Stejskal
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-03-17 12:41 UTC by Vladimir Voskresensky
Modified: 2010-04-06 08:40 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
log (gzipped) of long rescan after commit (452.82 KB, application/x-gzip)
2010-03-19 21:19 UTC, Vladimir Voskresensky
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Voskresensky 2010-03-17 12:41:11 UTC
I've started IDE with clean userdir and waited till everything is scanned (2 scans):
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 112 source roots took: 90516 ms (New or modified files: 3087, Deleted files: 0)
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 21 source roots took: 17052 ms (New or modified files: 223, Deleted files: 0)

Then I opened file, made small changes and committed (Mercurial->Commit) => full rescan with summary:
INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 589 source roots took: 74801 ms (New or modified files: 0, Deleted files: 0)
Comment 1 Vladimir Voskresensky 2010-03-17 12:44:47 UTC
next dummy commit => again reparse, faster but still annoying:

INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 589 source roots took: 17877 ms (New or modified files: 0, Deleted files: 0)
Comment 2 Vitezslav Stejskal 2010-03-18 07:54:27 UTC
It would be great to have a log file for an analysis, ideally one from the IDE started with -J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE. In general VCS operation that may change disk state run in the scanning-protected mode and make the IDE rescan (timestamp check) suspicious files/roots. We can't really avoid this, but can look at the efficiency. What exactly did you commit (how many files?, from how many source roots?, etc). Thanks
Comment 3 Vladimir Voskresensky 2010-03-19 21:19:59 UTC
Created attachment 95465 [details]
log (gzipped) of long rescan after commit

todays build, clean userdir, opened cnd modules and some other SunStudio modules.
committed two files (fixed issue #182245) => 40 sec of wait after commit :-(
Comment 4 Vladimir Voskresensky 2010-03-26 12:39:03 UTC
Due to this issue I'm using commits from terminal and (it's interesting) after going back to IDE it refreshes badges much much faster (about 4 secs) and without any logging messages (I use 
-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE)

But when I do the same in IDE I again have long scanning which makes mercurial integration unusable for commits.
FINE [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Invalidating source: org.netbeans.modules.parsing.api.Source@145ff3e[mimeType=text/xml, fileObject=MasterFileObject[/net/d-espb04-127-81/export/devarea/trunk/dlight.remote/nbproject/project.xml@1bb5348:1e59e9b,valid=true], document=null due to RootsWork
FINE [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Finished RefreshWork@cf4252[followUpJob=false, checkEditor=false, suspectFilesOrFileObjects=[Pair[/net/d-espb04-127-81/export/devarea/trunk,false]] in 41 094 ms with result Done
Comment 5 Vitezslav Stejskal 2010-03-29 14:40:26 UTC
Currently all VCS plugins treat 'commit' as a disk modifying command and as such request rescan. There are two question - 1. I this necessary/correct? 2. How much is rescanned?

re. #1 - AFAIK commit commands can modify contents of files due to VCS keyword substitutions, but these substitutions typically do not change file semantics and therefore have no impact on indexed data.

re. #2 - From the log file excerpts you posted here it looks like that whole trunk (eg. all source roots) are rescanned (more precisely the up-to-date check runs on all source roots). This sounds very inefficient after committing a single file. I'd assume that just the source root containing the file (plus maybe other source roots referring to the file) will be rescanned.

I'll keep investigating this further.
Comment 6 Ondrej Vrabec 2010-03-30 10:38:36 UTC
add 1)
- due to keywords substitution
- subversion specific: when you 'svn delete' a folder it won't be actually removed from disk until you commit the deletion (svn commit also modifies folder tree).

... fixed in cdev #3066a9f92d55 - from now on commits do not run under the Indexing Bridge and do not trigger the reindexing directly (however any commit may result in a sequence of FS events)

vita: i guess you can close this one, just try if the fix works.
Comment 7 Vladimir Voskresensky 2010-03-30 21:31:56 UTC
I can confirm: now it works without long full rescan.
Comment 8 Quality Engineering 2010-04-02 05:17:17 UTC
Integrated into 'main-golden', will be available in build *201004020200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/3066a9f92d55
User: Ondrej Vrabec <ovrabec@netbeans.org>
Log: Issue #182192 - mercurial->commit from IDE force full rescan
Comment 9 Vitezslav Stejskal 2010-04-06 08:40:12 UTC
(In reply to comment #6)
> 
> ... fixed in cdev #3066a9f92d55 - from now on commits do not run under the
> Indexing Bridge and do not trigger the reindexing directly (however any commit
> may result in a sequence of FS events)
> 
> vita: i guess you can close this one, just try if the fix works.

Thanks to Ondra for fixing this and Vladimir for testing.