This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 210200 - ER complains about scanning UI
Summary: ER complains about scanning UI
Status: NEW
Alias: None
Product: editor
Classification: Unclassified
Component: Parsing & Indexing (show other bugs)
Version: 7.2
Hardware: PC Linux
: P1 normal with 4 votes (vote)
Assignee: Tomas Zezula
Keywords: UI
: 210199 (view as bug list)
Depends on:
Reported: 2012-03-27 14:12 UTC by Svata Dedic
Modified: 2017-11-16 18:04 UTC (History)
112 users (show)

See Also:
Exception Reporter: 186344

stacktrace (1.91 KB, text/plain)
2012-10-02 11:50 UTC, schkovich
stacktrace (1.46 KB, text/plain)
2013-03-13 15:21 UTC, paolosca

Note You need to log in before you can comment on or make changes to this bug.
Description Svata Dedic 2012-03-27 14:12:36 UTC
Reporters are complaining that:
* they do not know why scanning is done
* scanning is done at the start
* scanning blocks their work

Better messages (e.g. "Checking for changed files" (upon project open), "Scanning added or removed sources", "File (VCS) changes detected, scanning for changes"). May be quite hard, requires cooperation from Parsing/path API users.
Comment 1 Svata Dedic 2012-03-27 14:13:55 UTC
*** Bug 210199 has been marked as a duplicate of this bug. ***
Comment 2 Tomas Zezula 2012-06-11 18:57:08 UTC
In fact in dev (NB 7.2) the situation is much better as the scan does not block editor features.
However I agree with Svata that the generic message should be replaced by some more detailed messages (at least in case where it's possible). I've added UI to cc as there was quite long discussion how the "Scanning project" should be renamed in NB 7.2 where it runs on background.
Comment 3 miked_187 2012-07-11 18:00:07 UTC
there is more to this than just "the scan does not block
editor features." as the scanning also hammers available system resources and while the editor might be available the system is so overloaded that the now available editor is unusable since the system isn't responding.  This scanning problem has been going on since pre-6.5 iirc, is there not a viable fix at some point to actually put the issue to bed??  Simply changing the messaging is insufficient.
Comment 4 Petah 2012-07-24 23:21:46 UTC
I agree, this bug has plagued me for many, many versions. Can't we just get it right?

Even though scanning is in the background now, it maxes CPU, and make NB unresponsive.

It takes hours on a cold start to scan my projects. And my projects are not 'that' big.

Some indication on what files NB is currently scanning would be nice. If I could identify the files that NB has problems with, I could remove them from my project.

Maybe a file parsing timeout could be added? If NB hasn't finished scanning a file in X seconds, then abort and ignore it.
Comment 5 petrabarus 2012-07-25 00:11:09 UTC
(In reply to comment #4)
> Some indication on what files NB is currently scanning would be nice. If I
> could identify the files that NB has problems with, I could remove them from my
> project.

I agree with this.

At first NB scanned my project smoothly. But after I accidentally put few 20MB JPEG files into my project's git repo, the NB scanning kept getting stuck. It was just a wild guess, I removed those JPEG files from my project and whole git history. Afterward the NB scanning got back to normal. (still, it's just a wild guess, It would be good if anyone could reproduce this bug)

Yeah, I think NB should show indicator on what files it is currently scanning.
Comment 6 Petah 2012-07-25 00:30:36 UTC
I have massive git folders, (multiple GB), and watching the Windows Resource Monitor, I can see Netbeans does take a while to scan them.

Should it even be scanning .git folders anyway?
Comment 7 Tomas Zezula 2012-07-25 08:23:00 UTC
>Should it even be scanning .git folders anyway?
Definitely not. The scanning uses VisibilityQuery API which VCS plugins implement or at least should implement to tell the scanning the the VCS metadata folder should be ignored. Works fine for mercurial, I will check the Git (probably the Git implementation forgot to do it) and create an issue on Git support.
Comment 8 Petah 2012-07-25 08:39:43 UTC
I normally disable the git plugin because of other issues. My projects are built from many git repos, the git plugin seem to lock some git files, and never release them. This causes issues when I want to move, rename, or delete a folder with a git sub repo in it.

I did submit a bug about that, but got the "wont fix" response.
Comment 9 miked_187 2012-07-25 16:30:19 UTC
there should be some way to specify which folders should be included, or which folders should be ignored.  I have a large static codebase that NB chokes on each time that there is no reason for it to be scanning over and over, I should have the ability to manually scan this subfolder area as needed when those libraries update.  git, svn, libraries, etc should be able to be ignored
Comment 10 dig 2012-09-03 20:47:19 UTC
My issue has been identified as a duplicate of this issue but its not an 'enhancement' it is totally broken for php projects:

* Scanning never completes, even if it eventually gets to 100% it doesn't finish.
* Code complete never shows anything for PHP projects (because scanning isn't complete?)
* Netbeans uses more system resources than seems sensible while its stuck on background scanning.

Neatbeans 7.2
Comment 11 Tomas Zezula 2012-09-04 07:01:29 UTC
to dig: Please fill a issue on PHP/editor.
It's good if you attach the self profile to it.
Comment 12 Marian Mirilovic 2012-09-19 12:56:36 UTC
227 reports, half of them came from 7.2
Comment 13 schkovich 2012-10-02 11:50:15 UTC
Created attachment 125219 [details]

Product Version: NetBeans IDE Dev (Build 201210020001)
Updates: Updates available
Java: 1.7.0_04; Java HotSpot(TM) 64-Bit Server VM 23.0-b21
System: Linux version 3.2.0-31-generic running on amd64; UTF-8; en_US (nb)
User directory: /home/goran/.netbeans/dev
Cache directory: /home/goran/.cache/netbeans/dev

After closing PHP project I canceled background scannning of the closed project. I expected gracefull stoping of the background. Actually i received message that IDE cannot interupt background process.
Comment 14 Marian Mirilovic 2012-11-05 10:06:59 UTC
369 duplicates, some reported against 7.3 Beta ... definitely P1

Any reason this is treated as ENH ? 

See for example :
Comment 15 Svata Dedic 2012-12-03 10:52:35 UTC
Originally, the attached report contained manually selected reports, which indicate that users do not understand what's going on or purpose of the background scan. But over time, the Exception reporter attached number of unrelated new reports to this issue.

I went through some (especially 7.3betaX) and tried to categorize them. 

These reports are invalid (I'll duplicate them to an already closed bug/report):

* Slow remote filesystem: 634547, 632314, 631194, 631096, 628449, 609434
* Invalid, data truncated: 627852, 626814, 621705, 621390, 620936, 632779
* N/A: 621327, 620027, 615064, 609301
* Normal work (lot of files, +- adequate time): 634146, 628436, 627663, 618991, 614607, 612553
* Cancelled shortly (< 2 mins) after scan: 633367, 632764, 631486, 631234, 629832, 629599, 629227, 628950, 628131, 627837, 626869, 626742, 626479, 619683, 616550, 616540, 620894
* OOM: 627353, 623757

I'll create separate issues for other reports, which contain meaningful info. Short summary:
* CSS parsing: 632835, 629654
* TLIndexer: 632740, 632519, 631172, 630874, 627478, 629050, 625880
* HTML CSS hints: 629949, 628928, 627116, 627108, 627009, 623054, 622357, 634323, 614169, 613921, 605429
* recursive directory (in 7.3 !!): (629944, 628004 - same user), 626978, 626971, 630542
* CSS cycle detected: excrep = 194833: 627994, 633197
* JS parsing: 626780, 616478, 614879; probably OK, pls check
* HTML parsing: 619159
* JPA processing: 617743
* PHP slow indexing: 624035
* APTUtils triggers backround parsing: 618502

The following issues were identified:
* Deadlock in REST services; report as issue #223087
* DataObjects may be created needlessly for files that might contain embeddings; filed as issue #627478
* indexer's vote to reindex all should be recorded and printed when scan is cancelled. Some reports indicate that only a fraction of resources is modified, while processing the source root still takes a lot of time as if all resources were scanned.
* better timers for indexing of embeddings: the time needed for parsing the outer content was not included into the statistics.

I recommend to ignore 7.2.x reports (I'll pick up and process some samples from the set), as based on past rounds of 'scan cancel' analysis, PHP, JS and possibly even Groovy took some actions to improve their code. Reports from past releases are becoming less relevant.
Comment 16 Exceptions Reporter 2012-12-06 07:01:55 UTC
This bug already has 450 duplicates 
Comment 17 Exceptions Reporter 2012-12-21 15:17:09 UTC
This bug already has 500 duplicates 
Comment 18 paolosca 2013-03-13 15:21:42 UTC
Created attachment 132571 [details]

While working the processor keeps running at 100%, background scanning seems locked and even if I close the IDE the java process is still running and I have to kill it manually.
This happens every time I use NetBeans for more that 20 minutes.
Note I have the same problem with OpenJDK 7 and OracleJDK 7.
See bug #227421 for the jstack output.
Comment 19 Exceptions Reporter 2013-05-06 18:40:42 UTC
This bug already has 950 duplicates