This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
In the past, Netbeans' memory usage was a concern. Nowadays, I've got 24-32GB of memory to play with depending on my machine. In such a configuration, I am expecting Find Usages (CTRL+F7) and Go to Implementation CTRL+ALT+B to be nearly-instantaneous. By comparison, Find Usages is currently taking 7 seconds on average on the commercial project I am working on ... and if I repeat the exact same search a second time, I have to wait an additional 7 seconds. Expected behavior: Assuming no classes were modified, repeated searches should return immediately and related searches should be nearly instantaneous. In general, please do more caching.
On machine with enough RAM there is a trick: 1) create in-memory disk (i.e. 2Gb) 2) start netbeans with extra param: --cachedir /path/to/ram/disk Then you get instant queries.
(In reply to Vladimir Voskresensky from comment #1) > On machine with enough RAM there is a trick: That's true. I have a 1GB ram disk and I have moved both userdir and cachedir to it. The responsiveness of the NetBeans it way better with such configuration. But it should be also possible without any tricks, only with a better caching on the IDE side. The ramdisk wastes memory when I'm not using NetBeans.
(In reply to cezariusz from comment #2) > (In reply to Vladimir Voskresensky from comment #1) > > On machine with enough RAM there is a trick: > > That's true. I have a 1GB ram disk and I have moved both userdir and > cachedir to it. The responsiveness of the NetBeans it way better with such > configuration. But it should be also possible without any tricks, only with > a better caching on the IDE side. The ramdisk wastes memory when I'm not > using NetBeans. I agree. Just wanted to share my experience. Btw, I recommend you to redirect into RAM only cachedir, because userdir keeps settings and usually you'd like to restore IDE after reboot of Windows.
Btw, have you tried to run IDE giving it more memory, i.e. like netbeans -J-Xmx2G ? Usually internal cache sizes are ratio of memory given to JVM, not physical machine memory (which JVM can not use for heap due to -Xmx limit)
(In reply to Vladimir Voskresensky from comment #3) > Btw, I recommend you to redirect into RAM only cachedir, because userdir > keeps settings and usually you'd like to restore IDE after reboot of Windows. I know, I'm using a software which automatically mirrors ramdisk on HDD. > Btw, have you tried to run IDE giving it more memory, i.e. like netbeans -J-Xmx2G I don't have this parameter, because netbeans.conf says: # Note that default -Xmx is selected for you automatically. It's worth trying to overwrite the default value as it's quite low (1024m in my case). Can cache use non heap memory, which is unlimited by this parameter?
Tomas, is it possible to configure lucene to use out of heap memory?
Why do we need to go there? Would using out-of-heap memory result in better performance than simply modifying the launcher to allow -Xmx to surpass 1GB? I think the latter might be enough and would be much easier to test.
Hi Vladimir, sorry for later answer I am now fully occupied by work on NB 9.0. 1) The answer is a bit complicated. Lucene can be configured to use mapped ByteBuffer which is in fact non heap memory managed by OS kernel. The OS memory manager takes a care about page in the required pages and evict them when the physical memory is decreasing. We use the memory mapped ByteBuffer when the JDK supports ByteBuffer unmapping (the java.nio.DirectByteBuffer has cleaner() method returning the sun.misc.Cleaner) which holds for all Oracle JDKs 7+. 2) Yes the out-of-heap memory result in better performance compared to heap memory as the GC does not need to care about it. The heap size affects not only old gen GC but also young gc in card table scan time. 3) NB does some caching of hot indexes in RAM up to 5% of the heap, the constant can be changed by the -java.index.size property, for example: -J-Djava.index.size=0.10 sets the RAM cache to 10% of the heap. In addition to this cache some additional indexes are kept in SoftReferences which expire when they are not used for some time or the heap size decreases. 4) On the fast disks (Vladimir's RAM disk) or SSD disk the caching makes no sense as the SSD disk is just 4 times slower compared to RAM and disabling the cache improves the performance. The reason why is that to cache the index whole index has to be loaded into the RAM while performing the query (if the query is not greedy) just loads several pages from the term dictionary and few from the document store. So on Windows and Mac where the user dir is located on SSD the -J-Djava.index.useMemCache=none. On Linux (Solaris) which have 1024 opened files per process it may be problematic as the indexes needs to be closed and reopen which is expensive. Unfortunately it's impossible in Java to find that the volume is SSD to do this automatically but I am working on some heuristic. But I doubt that anything from it will help much as the index query time is very small part of the find usages. As an example I am attaching an screenshot of finding usages of Index in ~100 projects which took 8.3s from which the time spent in index was 46ms. Probably most useful is the -J-Djava.index.useMemCache=none on SSD drive, tested on Mac not sure about other OSs. If anyone will try on Windows it would be great.
Created attachment 154254 [details] Find usages snapshot screenshot
Thomas, Any way to cache the "org.netbeans.modules.parsing.nb" results? It seems to be responsible for at least 50% of the cost.
Created attachment 154255 [details] Find usages snapshot screenshot 2
Hi, unfortunately the "org.netbeans.modules.parsing.nb" does only the DataObjectEnvFactory.runPriorityIO() (yellow on screenshot) which is a wrapper to run a Runnable as priority IO operation, the filesystems do not schedule any background task while the Runnable is running. It does not do any calculation, it just runs the FindUsages in my case JavaWhereUsedQueryPlugin (blue) which does the usages calculation . In this time the data from class index was already retrieved by FindUsages and Find usages finds the methods in sources which are using the "Index" most of time is spent in attribution of the sources (CC.toPhase - 3.2sec). Maybe some IO can be minimised there. Please can you attach your snapshot as I did the test on NetBeans Module Projects which are a nit specific (big number of relatively small projects)? The NB user dir is on disk hard drive or SSD? Thanks
Sorry for the late response. The Netbeans userdir is running on an SSD. For reference, when I run Find Usages or CTRL+LMB click on a class in IntelliJ IDEA it is instantaneous (sub-100ms). Under Netbeans Find Usages takes 7 seconds, and CTRL+LMB takes 3 seconds. I gave Netbeans an initial heap of 3GB and a max heap of 4GB. It never surpasses 1GB usage. For some reason as it nears 1GB something causes it to garbage collect leading me to believe that you're blowing away the cache prematurely.
Created attachment 154993 [details] IDE snapshot of Find Usages Here is the snapshot you requested. Let me know if it helps.
Sorry for later answer, here is what I found by analysing the sample. The time spent in the index search is 109ms. There is no reason to make anything faster, as 109ms is very small time. The rest is spent in the refactoring and support. One thing which is JaveRefactoringPlugin.getClassPathInfo() which transitively ends in Maven RepositoryForBinaryQueryImpl for 2,4s - uselessly as the usages are searched only in project sources. The ~3sec source file resolution, I am not sure if all the toPhase are needed. So step one should be fixing the maven problem which eats uselessly 1/2 time. Step to evaluate the toPhase calls in JavaRefactoringPlugin.
Tomas,What are the next steps for this issue? Does it need to be redirected to the Maven component?
Is anyone looking at this issue?