This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 185853 - Memory leak in Ant
Summary: Memory leak in Ant
Status: RESOLVED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Ant (show other bugs)
Version: 6.x
Hardware: PC Linux
: P2 normal (vote)
Assignee: Jesse Glick
URL:
Keywords: PERFORMANCE
: 186367 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-05-10 16:30 UTC by Tomas Mysik
Modified: 2010-05-19 13:51 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
jmap -permstat (102.44 KB, text/plain)
2010-05-10 16:30 UTC, Tomas Mysik
Details
Screenshot of profiler root references to the one Project instance I could find (34.43 KB, image/png)
2010-05-11 19:24 UTC, Jesse Glick
Details
Diagnostic (1.65 KB, patch)
2010-05-11 20:25 UTC, Jesse Glick
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Mysik 2010-05-10 16:30:33 UTC
Created attachment 98726 [details]
jmap -permstat

It seems that there's a memory leak in Ant - I got OOME in PermGen after I built several times a few NB modules - unfortunately no specific steps to reproduce. I have a heap dump (I can provide it), Tomas Hurka is investigating it. Also, I'm attaching output of "jmap -permstat". Petr Nejedly can comment more if needed, thanks.

Product Version: NetBeans IDE Dev (Build 100507-2f0ce976f254)
Java: 1.6.0_20; Java HotSpot(TM) 64-Bit Server VM 16.3-b01
System: Linux version 2.6.32-22-generic running on amd64; UTF-8; cs_CZ (nb)
Comment 1 Jesse Glick 2010-05-10 17:24:06 UTC
If you have a heap dump then presumably you could determine what is actually leaking and from where, using minimal GC root references.

Your jmap output seems to show everything dead.

I do not know of any recent changes of note in either the Ant integration or NB module build scripts that could have caused a recent regression. Of course if there are reliable steps to reproduce from scratch I can investigate.
Comment 2 Petr Nejedly 2010-05-10 21:42:27 UTC
jmap claims dead classloaders all the time. The log is there as the initial hint - there were actually 7 live AntBridge loaders, each of them eating around 10MB of permgen, each referencing like 1100 classes.
The heap dump should contain all the info, though retrieving it is far from selecting a class loader and asking the profiler for a root path - profilers usually fail at this kind of class leaks, NB profiler notwithstanding.

This is not incomplete, we have all the evidence, just not fully analyzed yet.
Comment 3 Tomas Mysik 2010-05-11 08:08:30 UTC
BTW for anyone interested in the heap dump, it can be found here:
http://dl.dropbox.com/u/1895361/heapdump-1273506693359.hprof.tar.gz
Comment 4 Petr Nejedly 2010-05-11 12:11:22 UTC
Out of those 7 AntBridge$MainClassLoader, 3 are reachable even through usual reference chain and visible in the NB profiler.
Those 3 loaders are all reachable from the same Project instance, through AntRefTable, through ComponentHelper$AntTypeTable, through (still the same instance) MacroDef$MyAntTypeDefinition.

The funny thing is that one class loader references the other class loaders through AuxClassLoader in a chain:
AB$MCL#7 -> ACL#6 -> AB$MCL#7
                  `> ACL#5 -> AB$MCL#5
and all of the AB$MCL class loaders define roughly the same 1100 classes, so AB$MCL#7 in fact "sees" 3 different implementations of, say, TaskContainer
(classes #25685, #23804, #26997, there are also "versions" #14626, #17052, #23450, #10618 in the JVM, loaded by the remaining 4 AB$MCL.

Now, regarding those 4 "unreachable" class loaders:
I somehow recall that it might be possible to run to permgen OOME condition when there are collectable objects on the permgem - that the permgen outage alone is not a trigger for a full GC. The same as you can run out of file handles or graphics contexts if you consume (and free) them rapidly without causing a heap collection. But I would have to verify this and a fixed heap walker might tell us earlier.
Comment 5 Jesse Glick 2010-05-11 18:29:31 UTC
(In reply to comment #4)
> Out of those 7 AntBridge$MainClassLoader

Generally there should only be one MainClassLoader active at a time. Currently it is permitted for this to be collected and recreated, though this behavior could be changed by making AntBridge.antInstance a strong reference if that seems helpful. (It is always recreated when you enable modules or change some things in Ant global settings.) Anyway, the question is not so much how many of them there are, as to who holds references to them.

> Those 3 loaders are all reachable from the same Project instance

This is uninteresting. The question is who is holding on to a Project after it has finished running. No references should remain.

> AB$MCL#7 -> ACL#6

Not sure what you mean by this; MainClassLoader does not reference AntClassLoader. Or what is "ACL"?
Comment 6 Jesse Glick 2010-05-11 19:24:47 UTC
Created attachment 98798 [details]
Screenshot of profiler root references to the one Project instance I could find
Comment 7 Petr Nejedly 2010-05-11 19:39:35 UTC
> The question is who is holding on to a Project after it has finished running. 
Oh, I thought it is an IDE Project without giving it too much attention. So it is an Ant internal class. Anyway, you answered this yourself.

>Not sure what you mean by this; MainClassLoader does not reference
> AntClassLoader. Or what is "ACL"?

AuxClassLoader (mentioned few lines higher).
Now that you have the heap dump, you see it yourself (show roots for AB$MCL#5).
You claim that there should be only one AB$MCL active, but the Project clearly references 3 of them somehow...
Comment 8 Jesse Glick 2010-05-11 19:41:13 UTC
What I see there looks like some kind of bug in RequestProcessor; I am not sure why ExplorerManager.selectionProcessor should be holding a reference to a task from BridgeImpl.RP. Anyway, I can try to make all such tasks at least not refer to any Project; maybe that will help, though I cannot really tell without a way to reproduce.
Comment 9 Jesse Glick 2010-05-11 20:08:42 UTC
It is normal for there to be several AuxClassLoader's per MainClassLoader - one per NB module with ant/nblib/$cnb.jar. I don't really follow what the MCL #7 -> MCL #5 link is about; URLClassLoader.acc is somehow set to something based on previous ACLs.
Comment 10 Jesse Glick 2010-05-11 20:09:52 UTC
core-main #c57ab70402d1
Comment 11 Jesse Glick 2010-05-11 20:24:39 UTC
(In reply to comment #9)
> I don't really follow what the MCL #7 -> MCL #5 link is about

Nor can I reproduce it, using the attached diagnostic.
Comment 12 Jesse Glick 2010-05-11 20:25:14 UTC
Created attachment 98802 [details]
Diagnostic
Comment 13 Tomas Mysik 2010-05-12 08:25:51 UTC
(In reply to comment #8)
> I cannot really tell without a way to reproduce.

What I was doing - I was developing [1] NB modules for the whole day, this happened after about 9 hours.
Also, maybe it could be related to JDK version - few days ago I updated from 1.6.0_16 to 1.6.0_20 (via update manager).

I tried to reproduce yesterday, what I did exactly:
- I started VisualVM and attached it to my NB instance
- Sampler > Memory; PermGen histogram + Deltas

The delta was OK for several hours but later in the afternoon I noticed that the delta is about 16 MB. I have a heap dump if anyone is interested, please, let me know, I will provide it.

[1] edit file, save, rebuild module
Comment 14 Jesse Glick 2010-05-12 14:26:06 UTC
(In reply to comment #13)
> I tried to reproduce yesterday

Before or after the supposed fix? (hg debugancestor is useful for checking)

> I have a heap dump if anyone is interested

Only if (a) it was taken with an IDE including c57ab70402d1, (b) it shows a similar issue of a live ant.Project when no process is running.
Comment 15 Tomas Mysik 2010-05-12 14:43:59 UTC
(In reply to comment #14)
> Before or after the supposed fix? (hg debugancestor is useful for checking)

Before.

> Only if (a) it was taken with an IDE including c57ab70402d1, (b) it shows a
> similar issue of a live ant.Project when no process is running.

No.
Comment 16 Quality Engineering 2010-05-13 05:03:26 UTC
Integrated into 'main-golden', will be available in build *201005122200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/
User: 
Log:
Comment 17 Tomas Pavek 2010-05-19 13:51:14 UTC
*** Bug 186367 has been marked as a duplicate of this bug. ***