This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 166800 - [67cat] Opening of netbeans projects from OpenJDK sources needs ~2 hours
Summary: [67cat] Opening of netbeans projects from OpenJDK sources needs ~2 hours
Status: VERIFIED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P2 blocker (vote)
Assignee: Dusan Balek
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2009-06-09 17:22 UTC by ulfzibis
Modified: 2009-11-11 16:10 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
log files (36.35 KB, application/x-compressed)
2009-06-28 18:19 UTC, ulfzibis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ulfzibis 2009-06-09 17:22:46 UTC
[ BUILD # : RC1 ]
[ JDK VERSION : 1.6.* ]

Refer to http://nb-openjdk.netbeans.org/

I think, it should be possible, that Netbeans only scans sources of
interest, listed in projects Source Packages node instead of whole
bunch of JDK sources. It should rely on the just compiled classes
from the bootstrap JDK.
Comment 1 Milos Kleint 2009-06-09 18:41:37 UTC
not really project infrastructure related, reassigning to ant/freeform, the described problem could be a configuration
problem on the side of the openjdk project.
Comment 2 David Strupl 2009-06-26 09:01:57 UTC
Dusan, did you mention that you are able to open JDK sources in ~150s? Seems something is broken in the project
configuration here if it takes 2 hours ...
Comment 3 Dusan Balek 2009-06-26 09:41:33 UTC
Could you please try to reproduce the issue using current dev build (containing the following changes
http://hg.netbeans.org/main-silver/rev/3575a6304958
http://hg.netbeans.org/main-silver/rev/3109509e9ee4)?

Thanks.
Comment 4 ulfzibis 2009-06-26 10:09:44 UTC
Could you please add the link here, when the build is ready for download?
For an external than me it seems complicated to find it otherwise.
Comment 5 ulfzibis 2009-06-26 14:21:23 UTC
Is there any chance to log the scan time, without sitting 2 hours in front of my screen, watching the progress bar to
finish?
Comment 6 David Strupl 2009-06-26 14:39:04 UTC
Our team's building maching is e.g. here
http://bertram.netbeans.org/hudson/job/jet-main/
--- the build is under "last successfull artifacts".

You can try to enable logging, e.g. some of the following in the etc/netbeans.conf

-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE

#-J-Dorg.netbeans.modules.parsing.impl.indexing.PathRecognizerRegistry.level=FINE \
#-J-Dorg.netbeans.modules.parsing.impl.indexing.PathRegistry.level=FINE

#-J-Dorg.netbeans.api.java.classpath.ClassPath.level=FINE
#-J-Dorg.netbeans.modules.parsing.impl.indexing.FileObjectCrawler.level=FINER \
#-J-Dorg.netbeans.modules.java.source.indexing.JavaCustomIndexer.level=FINE
#-J-Dorg.netbeans.modules.csl.core.PathRecognizerImpl.level=FINE \


#-J-Dorg.netbeans.modules.parsing.impl.indexing.TimeStamps.level=FINE \
#-J-Dorg.netbeans.modules.parsing.impl.event.EventSupport.level=FINE
Comment 7 ulfzibis 2009-06-26 14:56:26 UTC
Dmitri, thanks.

Does it harm to enable all given options at same time. I don't want to run the 2-hour-scan 9 times in worst case.
Are these options also valid for NB 6.5, to make the comparison?
Comment 8 ulfzibis 2009-06-27 08:42:58 UTC
Can't still find the changes in any build on http://bertram.netbeans.org/hudson/job/jet-main/changes
Comment 9 Marian Mirilovic 2009-06-27 13:38:55 UTC
Ulf, please test build from daily builds :
http://bits.netbeans.org/dev/nightly/2009-06-26_14-01-07/

it contains fixes from dbalek and please let us know. Your feedback is highly appreciated. Thanks in advance.
Comment 10 ulfzibis 2009-06-27 15:04:30 UTC
Please give me a hint about my logging question above.
Comment 11 Marian Mirilovic 2009-06-27 22:51:15 UTC
please use following for now :
-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE
Comment 12 ulfzibis 2009-06-28 18:17:00 UTC
Congratulation !!

I'm sure you have caught the BIG FISH ... 28 seconds against >2 hours for scanning the OpenJDK project. BRAVO!
Also Issue 166668 seems to be obsolete now.

Hopefully we can have this patch for 6.7 release ???
But attention:
Issue 167751
Comment 13 ulfzibis 2009-06-28 18:19:21 UTC
Created attachment 84091 [details]
log files
Comment 14 ulfzibis 2009-06-28 20:36:27 UTC
Additionally memory consumption was only ~60M against 500M before (I guess, it would have been more, if limit wasn't set
to 500M).
Comment 15 Milan Kubec 2009-06-29 08:00:28 UTC
Should this issue be closed? Or at least reassigned to java, it seems that it has nothing to do with freeform project
itself.
Comment 16 David Strupl 2009-06-29 08:12:47 UTC
I is marked with the 67patch-candidate status whiteboard.

Ulf, Dusan, should we close this report as fixed?
Comment 17 Dusan Balek 2009-06-29 08:55:58 UTC
Marking as fixed in 6.8
Comment 18 Petr Blaha 2009-07-02 13:10:09 UTC
I'm closing as verified based on ulfzibis comment.
Comment 19 pgebauer 2009-07-03 19:34:40 UTC
I have tried to port the bugfix of this issue to the release67_fixes branch. It seems that above mentioned changesets ( http://hg.netbeans.org/main-
silver/rev/3575a6304958 ,  http://hg.netbeans.org/main-silver/rev/3109509e9ee4 ) are not complete set of changes those should be ported. After 
application of mentioned changes to the local release67_fixes repository, the compiler cannot find the field org.netbeans.modules.java.source.indexing.CompileWorker.ParsingOutput.modifiedTypes which was introduced by another changeset ( http://hg.netbeans.org/main-silver/rev/e040650d2d7c ) not mentioned here. Another two changesets ( http://hg.netbeans.org/main-
silver/rev/ea76467e7766 and http://hg.netbeans.org/main-silver/rev/4916a251224c ) have been committed to the repository between e040650d2d7c and 3575a6304958. Should they all be ported into the release67_fixes branch or just part of them? I would appreciate a clarification or help. Thank you.
Comment 20 David Strupl 2009-07-07 14:43:25 UTC
Unfortunatelly Dusan is not here this week. I suggest that we wait for him to return - if we cannot wait I suggest to
port *all* the changesets you mentioned since the 2 last ones are harmless (one logging and one other perf improvement).

This issue has to be very carefully tested on branch release67_fixes before it is released to make sure that we merged
all needed files correctly and did not introduce any regression.
Comment 21 Dusan Balek 2009-07-13 09:59:37 UTC
The easiest way to port the bugfix would be: port both above mentioned changesets
(http://hg.netbeans.org/main-silver/rev/3575a6304958 and http://hg.netbeans.org/main-silver/rev/3109509e9ee4) and
manually change both occurrences of 'previous.modifiedTypes' to 'previous.root2Rebuild'.

Anyway, careful testing of this issue on branch release67_fixes before it is released would be recommended ;-)
Comment 22 David Strupl 2009-07-13 10:01:58 UTC
Dusan, can you please do it? I mean the porting to the branch ... Thanks.
Comment 23 Dusan Balek 2009-07-13 14:25:53 UTC
The fix has been ported into the release67_fixes repository.

http://hg.netbeans.org/release67_fixes/rev/215ceaabcf76
http://hg.netbeans.org/release67_fixes/rev/9e0009873ad9
Comment 24 ulfzibis 2009-07-13 14:42:52 UTC
Good news. :-)
This makes me happy, because when 6.7.1 will be out, I too should be able to use official release for my normal work and
can get rid of using dev builds.
Comment 25 Jiri Prox 2009-07-20 14:39:33 UTC
verified in 6.7.1
Comment 26 ulfzibis 2009-11-11 16:10:30 UTC
Startup performance has repeatedly increased. :-)

Build 200910230201:
Opening projects: 30 seconds
Initial scan: ~1-2 minutes (was ~2 hours: Issue 166800)