Created attachment 58249 [details]
Logfile with tons of exceptions

The call causing problem is "com.sun.tools.javac.model.JavacElements.getTypeElement()" which in the end results in ZIP
exception in "org.netbeans.modules.java.source.parsing.CachingArchive.createMap()" method.
From the first look it doesn't seem like a profiler problem but I will investigate further

The root cause is definitely depleting the pool of file descriptors. It seems it only happen on some linux distributions
where the number of opened files is limited to 1000 - in my case Ubuntu.
The excessive usage is caused by the fact that we in profiler use SourceForBinaryQuery to get the list of library only
classes from the classpath. AFAIK, it's not possible to get that list by any other means.
For now, the workaround is to increase the limit on open files.

fixing this issue will require substantial changes in the profiler code related to how a project classpath is
constructed and used. this is definitely not achievable for 6.1
requesting waiver

*** Issue 133482 has been marked as a duplicate of this issue. ***

too big IDE :(.  

*** Issue 138334 has been marked as a duplicate of this issue. ***

Not sure how this is related to the SourceForBinaryQuery. I tried this:
1. started profiling of java.j2seproject, took look on which files become open (through /proc/<id>/fd). One of them was:
"netbeans/enterprise5/modules/org-netbeans-modules-websvc-utilities.jar" which I used in further experiments.
2. I have augmented the "checkRead" method in the NB's security manager (o.n.bootstrap, TopSecurityManager) to show a
stacktrace each time someone wants to read the above file. Tried to profile java.j2seproject again. The interesting
stacktrace touching the given file was:
java.lang.Exception:
/usr/local/home/lahvac/src/nb/cos/nbbuild/netbeans/enterprise5/modules/org-netbeans-modules-websvc-utilities.jar
	at org.netbeans.TopSecurityManager.checkRead(TopSecurityManager.java:299)
	at java.util.zip.ZipFile.<init>(ZipFile.java:109)
	at java.util.zip.ZipFile.<init>(ZipFile.java:131)
	at org.netbeans.lib.profiler.classfile.ClassPath.<init>(ClassPath.java:181)
	at org.netbeans.lib.profiler.classfile.ClassRepository.initClassPaths(ClassRepository.java:453)
	at org.netbeans.modules.profiler.NetBeansProfiler.cleanupBeforeProfiling(NetBeansProfiler.java:1863)
	at org.netbeans.modules.profiler.NetBeansProfiler.access$2700(NetBeansProfiler.java:161)
	at org.netbeans.modules.profiler.NetBeansProfiler$3.doInBackground(NetBeansProfiler.java:1091)
	at org.netbeans.lib.profiler.ui.SwingWorker$2.run(SwingWorker.java:113)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:561)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:986)
3. I took a look at org.netbeans.lib.profiler.classfile.ClassPath and it is seems to be opening ZipFiles, but it is not
clear to me when it is closing them (maybe at the end of the profiling session?). I added debugging messages into
org.netbeans.lib.profiler.classfile.ClassPath.<init> to check when a ZipFile is opened and to
org.netbeans.lib.profiler.classfile.ClassPath.close to check when a ZipFile is closed. After running the profiler again,
I got ~978 messages about opening a ZipFile, but none about closing them. There are ~978 jar files in NetBeans, so it
seems to me that the profiler is trying to open all the .jar files, but does not close them soon enough (978 opened
files seems too much to me).

Would it be possible to verify my observations, investigate when are the files being closed and how to prevent opening
so much files concurrently? Thanks.

org.netbeans.lib.profiler.classfile.ClassPath caches ZipFile-s to avoid their repetitive creation in the code that reads classfiles from JARs. The ZipFile-s are 
closed when profiling finishes. 

It looks like the initial evaluation is incorrect and the problem can be fixed in profiler code.
I will see what can be done to decrease number of simultaneously opened jars.

Fixed.
changeset:   86802:a1dd11c30d9d
user:        Tomas Hurka <thurka@netbeans.org>
date:        Mon Jun 30 14:40:19 2008 +0200
summary:     bugfix #129931, do not open all jars from classpath in the ClassPath constructor, open jars only when needed by profiler and use LRU cache for 
already opened jars

Integrated into 'main-golden', available in NB_Trunk_Production #294 build
Changeset: http://hg.netbeans.org/main/rev/a1dd11c30d9d
User: Tomas Hurka <thurka@netbeans.org>
Log: bugfix #129931, do not open all jars from classpath in the ClassPath constructor, open jars only when needed by profiler and use LRU cache for already opened jars

*** Issue 138317 has been marked as a duplicate of this issue. ***