This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
The whole netbeans.org CVS tree is scanned to the depth 3 when sources of some module belonging to this tree are requested. This is very expensive operation that requires a lot of I/O while scanning the disk content and a lot of CPU time when configuration files are processed (both XML and properties). We should try to narrow the amount of procesed data. According to Jesse: One trick that would be useful *only* for developing netbeans.org modules which are in the standard build is to find standard modules in cluster.properties, since these could not depend on experimental modules anyway; if you initiate a scan from a module which is not in this list, then kill the cache and fall back to the current behavior. Another possibility is to reduce set of modules in source distribution to not contain things that are not part of standard distribution. This can help to users building on top of NetBeans that wants to have access to sources.
One of the most frequent complaints about the new NBM project support.
Some initial results from the unit test only: INITIAL Time to scan netbeans.org sources: 1871msec Directories traversed: 1324 XML files parsed: 433 in 710msec Time to scan suite + NB binaries: 475msec Time to scan netbeans.org sources: 2838msec Directories traversed: 1324 XML files parsed: 433 in 1173msec Time to scan suite + NB binaries: 477msec Time to scan netbeans.org sources: 1977msec Directories traversed: 1324 XML files parsed: 433 in 770msec Time to scan suite + NB binaries: 597msec Time to scan netbeans.org sources: 1814msec Directories traversed: 1324 XML files parsed: 433 in 715msec Time to scan suite + NB binaries: 851msec AFTER #62221 Time to scan netbeans.org sources: 2821msec Directories traversed: 586 XML files parsed: 197 in 1605msec Time to scan suite + NB binaries: 462msec Time to scan netbeans.org sources: 2337msec Directories traversed: 586 XML files parsed: 197 in 454msec Time to scan suite + NB binaries: 468msec Time to scan netbeans.org sources: 1590msec Directories traversed: 586 XML files parsed: 197 in 674msec Time to scan suite + NB binaries: 534msec Time to scan netbeans.org sources: 1588msec Directories traversed: 586 XML files parsed: 197 in 688msec Time to scan suite + NB binaries: 634msec Looks promising.
Need testtools too, so a bit more: Time to scan netbeans.org sources: 1206msec Directories traversed: 596 XML files parsed: 204 in 462msec Time to scan suite + NB binaries: 660msec
Have patch that appears to do the job. (This deals only with the IDE's scanning, not the scanning done by Ant tasks.) A request for the modules in a netbeans.org initially scans only those modules mentioned in cluster.properties, among ${clusters.list}: i.e. all modules in the standard IDE distribution, plus testing tool modules (which need to be scanned to make test code completion work). This avoids a lot of file I/O looking for matching dirs, as well as cutting the number of XML files that need to be parsed in half. Certain events trigger a full depth-3 scan, such as opening a module not in the standard distribution; this just expands the existing module list with the extra modules. Also, the module list in the default platform will not show extra modules (which is probably just as well). I did not change the depth-3 scan which (as of recently) occurs inside a ZIP file of netbeans.org sources when you associate this with a binary-only platform because (1) source downloads include only CVS modules required for stable modules, excluding many extra modules to start with (but leaving things like ant/docs and openide/looks); (2) this scanning is all inside a ZIP file, which I expect should be faster than a scan through physical folders. ModuleListTest now prints somewhat more useful summary statistics, which apply to the quick scan of stable modules only.
committed * Up-To-Date 1.121 apisupport/project/src/org/netbeans/modules/apisupport/project/NbModuleProject.java committed * Up-To-Date 1.19 apisupport/project/src/org/netbeans/modules/apisupport/project/layers/LayerUtils.java committed * Up-To-Date 1.38 apisupport/project/src/org/netbeans/modules/apisupport/project/ui/customizer/SingleModuleProperties.java committed * Up-To-Date 1.19 apisupport/project/src/org/netbeans/modules/apisupport/project/universe/ModuleList.java committed * Up-To-Date 1.25 apisupport/project/src/org/netbeans/modules/apisupport/project/universe/NbPlatform.java committed * Up-To-Date 1.13 apisupport/project/test/unit/src/org/netbeans/modules/apisupport/project/universe/ModuleListTest.java
Also for 5.1, applying a similar trick for the Ant tasks, which should yield some speed improvement for e.g. compiling or testing a module in the standard distro: committed Up-To-Date 1.30 nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java