This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 33979 - Expanding packages is slow and does not scale
Summary: Expanding packages is slow and does not scale
Status: CLOSED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Generic Infrastructure (show other bugs)
Version: 3.x
Hardware: All All
: P1 blocker (vote)
Assignee: issues@projects
URL: http://performance.netbeans.org/nonav...
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2003-05-28 18:21 UTC by Tomas Pavek
Modified: 2003-12-11 14:25 UTC (History)
0 users

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
snapshot of expanding o.n.m.form (another package already expanded); "sampler" method - suitable for relative time comparison (133.47 KB, application/octet-stream)
2003-05-28 18:27 UTC, Tomas Pavek
Details
snapshot of expanding o.n.m.form (with another package already expanded); "instrumentation" method - shows invocation counts, but distorts relative times proportions (348.18 KB, application/octet-stream)
2003-05-28 18:29 UTC, Tomas Pavek
Details
classes loaded during the very first expansion (9.74 KB, text/plain)
2003-05-28 18:33 UTC, Tomas Pavek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Pavek 2003-05-28 18:21:43 UTC
Expanding a package (under Packages node) is slow
and the time to expand rises quickly with the
number of files in the package. E.g. time to
expand org.openide.explorer (4 files) was 250ms,
org.openide.loaders (~50 files) took about 2.5s
(taken from the NetBeans "Projects" - Initial
Performance Report). Most recent measuring show
about 5sec to expand org.netbeans.modules.form
package (about 80 files) [build 200305230000 with
the form module integrated, jdk 1.4.1_02, Win2K,
PC 733MHz]. In NB 3.5 this was about 0.5sec.

This is one of the most important performance
issues found during the initial performance
measuring of Projects.

I'm filing this issue for java module, however
there can be more causes in various areas
(projects infrastructure, looks, other modules).
We may reassign the issue later, or create sub-issues.
Comment 1 Tomas Pavek 2003-05-28 18:24:52 UTC
Attaching profiling snapshots done in OptimizeIt on prj40_prototype
build 200305230000 (with form module integrated additionally). Tried
to expand org.netbeans.modules.form package from NB 3.5 (77 .java
files, 6 .form files, .properties files).
Comment 2 Tomas Pavek 2003-05-28 18:27:44 UTC
Created attachment 10493 [details]
snapshot  of expanding o.n.m.form (another package already expanded); "sampler" method - suitable for relative time comparison
Comment 3 Tomas Pavek 2003-05-28 18:29:42 UTC
Created attachment 10494 [details]
snapshot of expanding o.n.m.form (with another package already expanded); "instrumentation" method - shows invocation counts, but distorts relative times proportions
Comment 4 Tomas Pavek 2003-05-28 18:32:41 UTC
Both snapshots are taken after some package was already expanded, so
it is not the very first expansions - classes are loaded and static
initialization should be done. Looks like it makes no big difference
either, so not attaching the very first use snapshots. See the list of
loaded classes during the first expansion - it is really rather short.
Comment 5 Tomas Pavek 2003-05-28 18:33:19 UTC
Created attachment 10495 [details]
classes loaded during the very first expansion
Comment 6 Tomas Pavek 2003-05-28 18:37:51 UTC
According to very quick look at the snaphots, it seems the problem
lies in Looks, but probably the project infrastructure and java are
somehow guilty too. Further analysis is needed.

Can anybody from java/projects look at this quickly if it makes sense
to deal with deeper analysis already now, or things may change
significantly after rewriting to new Looks?
Thanks.
Comment 7 Vitezslav Stejskal 2003-05-29 08:26:54 UTC
It seems that time is spent mostly by doing two things:

1. constructing Lookup for ProjectMembers
2. creating LookNodes visualizing the ProjectMembers

The #2 will definitely change after rewriting projects for new Looks
API - although I can't say if it will be better or worse.

The #1 is IMO caused by inefficient implementation of
ProvidersNamespace class, this class is responsible for looking for
all ProjectEnvironmentProviders registered for specific MIME type (the
type of ProjectMember). It seems to me that the namespace is searched
always again _for each_ ProjectMember instance even if it is of the
MIME type for which results were already found. The worst thing is
that searching ends up by creating another set of PEPs.

There are also other time consuming places in this report (not so bad
as #1), but still:

- HostFileObject.isFolder - tries to discover if FileObject isn't
mount point
- MultiFileObject.getAttribute - it's really long way from MFO to some
real FileObject on XML layer or LocalFileSystem
- DataLoaderPool.findDataObject

Comment 8 Vitezslav Stejskal 2003-06-05 12:43:23 UTC
The evaluation is done (ETA=3). As far as we have to solve the #1 I am
 reassigning this to projects infra.
Comment 9 Vitezslav Stejskal 2003-11-26 13:17:59 UTC
As described in
http://www.netbeans.org/servlets/ReadMsg?msgId=619519&listName=nbdiscuss the
current work on projects prototype has been stopped.
Comment 10 Vitezslav Stejskal 2003-11-26 14:57:57 UTC
--> VERIFIED