This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
MDR has its own proxy class generation facility, which generates over a hundred JMI implementation classes each time NetBeans runs. To check this, I added a println statement to the top of org.netbeans.mdr.handlers.gen.HandlerGenerator.generateHandler(). Try adding a print line there and watch as classes mainly get generated during three performance-critical activities: 1. after the main window displays for the first time but before your CPU meter drops to zero; 2. during classpath scanning; and 3. the first time you open a class to view its members in the Explorer. Shouldn't these generated classes be created during the build like the rest of our classes? The Java language won't be changing any time soon (and certainly not between every IDE invocation!). Better yet, since javacore implements the same JMI interfaces that the generated handlers do, how about merging the functionality of these classes? It seems that a single, static implementation would perform much better and take less footprint. Also, a separate set of interface implementations are generated by MDR to initialize its repository files for the first time. MDR initialization could instead be run during installation, since MDR can be (or used to be) executable from the command line.
The interface implementations are generated dynamically not because we thought that java language will change often, but because: 1) we want to avoid regenerating all metamodel implementations if the MDR byte code generator is updated (we fix some bug, etc.) - this way the new version of MDR can be distributed using autoupdate without a need for regenerating all metamodel implementations installed on user's machine 2) the assumption was the byte code generation is not significanlty slower than loading classes from the disk (I see no indication in this issue that this is not the case) There is a separate issue filed regarding the MDR initialization during installation (see issue 45054).
re 1): you need to update users instalation anyway, either with bytecode generator or with new version of generated classes. there is no advantage in current approach IMO. re 2): have you measured this before you made this assumption? How many classes do we need to generate? How many classes of bytecode generator do we need to load to make this system working. IMO you can spent hundreds of milliseconds in generator (can be checked with some profiler). Of course more garbage is produced during this time than with usual classloading. Average time to load a class is less than 1 ms. Cf. 'jstat -class' or other tools to see how much time JVM spents in classloading.
re re 1) the advantage seems obvious to me - while in the case of dynamic bytecode generation the MDR module maintainer can just change the MDR module and all the other applications/modules build on top of it will automatically adapt the change - no additional work needed, in the static bytecode generation with every change to the MDR and the bytecode generator (bugfixing, etc.) we would have to notify everyone to regenerate their implementations. re re 2) I haven't measured it yet, but bytecode generation never caused a noticable delay and I never saw it as a significant performance blocker when profiling MDR and javacore. Anyway, note I haven't closed this issue as INVALID or WONTFIX. I am going to change MDR to store generated classes in the b-tree so that they do not need to be generated over and over again. The classes will be properly versioned so that if someone changes the generator or the metamodel, they will get regenerated again. Happy?
I am not quite convinced that this is the best approach but it is going to be an improvements ;-) So the assumption is that reading from btreestorage is comparable to our ProxyClassLoader. It might be. Wouldn't it be easier for testability too to dump them into normal files (and avoid special classloader)? OK, I do not have strong opinion here. If I recall correctly there were also some string containing method signatures for generated classes. I expect that data like these are not needed if we implement this so it can be another benefit.
Hmm, I have measured that the IDE spends ~1.5s in generateHandler during the startup with some project open. Note that mentioned 1.5s is on 2GHz NTB when I tried to reduce paralel threads running on the machine (If I just measured dusing currentTimeMillis on unmodified startup, I got nearly 3s) First few classes take most of the time as the generator is running interpretted and may be even JITted during execution. Removing the runtime generation step, we'll also save some time spent loading the generator classes and the memory used by the generator classes (~100kB jist in .class files which may consume much more once loaded).
Trying with build from Oct 4, JDK 1.5.0 on Solaris ULTRA 60 with sampling by Analyzer every 1ms. My startup with 2 project and soem Java editors takes ~95s and 3.260ms is spent in HandlerGenerator.generateHandler. These are 3% of total startup time. As Petr mentioned all the stuff related to runtime generation is almost 100kB of classes, it holds some data for generation and likely only small part of this system will be optimized by hotspot. It generates ~400kB for 88 classes in my case (probably couple more classes can be generated later). BTree database vs. ProxyClassLoader - once you dump the classes it is quite easy to package them and refer to them in your modules. If you want to store them in database you need to have special classloader and there will be some overhead related to maintainance of btree structures and versioning.
Dan is going to fix this.
Checking in handlers/gen/HandlerGenerator.java; /cvs/mdr/src/org/netbeans/mdr/handlers/gen/HandlerGenerator.java,v <-- HandlerGenerator.java new revision: 1.14; previous revision: 1.13 done Processing log script arguments... More commits to come... Checking in storagemodel/StorableObject.java; /cvs/mdr/src/org/netbeans/mdr/storagemodel/StorableObject.java,v <-- StorableObject.java new revision: 1.41; previous revision: 1.40 done Processing log script arguments... More commits to come... Checking in util/IOUtils.java; /cvs/mdr/src/org/netbeans/mdr/util/IOUtils.java,v <-- IOUtils.java new revision: 1.32; previous revision: 1.31 done
Would it be possible to move the caching logic out of the HandlerGenerator to its caller (BaseObjectHandler I guess)? Doing this change, you can spare loading the generator classes, which are huge. (with verification on, most of the gen package is still loaded, but even with verification off, we can spare 4 classes, 2 of them quite big).
Good idea!
Done. /cvs/mdr/src/org/netbeans/mdr/handlers/BaseObjectHandler.java new revision: 1.47; previous revision: 1.46 /cvs/mdr/src/org/netbeans/mdr/handlers/gen/HandlerGenerator.java new revision: 1.15; previous revision: 1.14
HandlerGenerator is not loaded in build from Oct 20, BaseObjectHandler.resolveClass takes 379ms