This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 194449 - Evaluate possible leaking class loaders (PermGen)
Summary: Evaluate possible leaking class loaders (PermGen)
Status: RESOLVED FIXED
Alias: None
Product: javaee
Classification: Unclassified
Component: Hibernate (show other bugs)
Version: 7.0
Hardware: All All
: P3 normal (vote)
Assignee: Sergey Petrov
URL:
Keywords:
Depends on:
Blocks: 191872
  Show dependency tree
 
Reported: 2011-01-18 20:01 UTC by Petr Jiricka
Modified: 2012-05-17 12:54 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Jiricka 2011-01-18 20:01:13 UTC
Umbrella bug 191872 contains several unrelated OOME problems, of which this is Hibernate-related:
http://statistics.netbeans.org/exceptions/exception.do?id=430922
Comment 1 Sergey Petrov 2011-01-26 17:42:59 UTC
report contain dump for 158Mb but 450+ is reported as max value, it looks most data is gc-ed at time of report and  it may not be memory leak. but may need to investigate more as I may miss something.
Comment 2 Sergey Petrov 2011-01-27 13:13:04 UTC
As it's PermGen, previous comment may mean nothing.
Comment 3 Sergey Petrov 2011-01-27 17:28:43 UTC
I'm not yet good in perm gen and dump evaluation, but dump show a lot of mysql classes loaded by 34 different instances of classloaders, most (about 30 in some cases and about 15 in another) are org.netbeans.modules.hibernate.util.CustomClassLoader. And it may mean there is a problem in hibernate.
HibernateEnvironmentImpl::getProjectClassLoader create new classloader on each access, wonder if it can be optimized here, any ideas?
On other side it's just one report without any steps but I'm going to play more with it.
Comment 4 Sergey Petrov 2011-01-31 11:12:08 UTC
Each hql querry execution create new classloader, each may hold jdbc classes and may not be released, there are two ways find why it's not released and try to keep 1-2 classloaders per project(i.e. try to cache instead of new creation each time) so the issue will effectively gone away. Still need more investigation if it's the as I'm not getting any PermGen yet.
Comment 5 Quality Engineering 2011-02-03 05:51:08 UTC
Integrated into 'main-golden', will be available in build *201102030000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/82425bdd0d91
User: Sergey B. Petrov <sj-nb@netbeans.org>
Log: #194449 just two places may cause to keep some references
Comment 6 Sergey Petrov 2011-02-09 14:27:07 UTC
From investigation driver is kept by DriverManager, each hql querry add one more driver, each "pojo from" add number of drivers also, each one is loaded by new classloader and as reference is here(in DriverManager) classloader isn't released it keep a lot of loaded jdbc classes definitions and duplicate on each access - >Permgen.

There is security issues with access and deregistering of drivers, for example even if I explicitly  register driver with DriverManager.registerDriver right after driver class loading in CustomJDBCConnectionProvider, I will not receive it back just in next line with DriverManager.getDrivers(); and same can't deregister next line and will receive securityexception as DriverManager is loaded by system classloader, CustomJDBCConnectionProvider is by OneModuleClassloader and driver class itself by HibernateCustomClassloader.
Also looks like even without explicit registration it's registered here anyway, not sure where. Googling for  "jdbc driver custom classloader" provider some interesting links also. Don't have a solution yet.
Comment 7 Sergey Petrov 2011-02-09 15:20:34 UTC
driver register itself in DriverManager in next call in nb code "clazz.newInstance()".
Comment 8 Petr Jiricka 2011-02-17 13:22:19 UTC
Cc'ing Jirka - Jirko, do you know how this works in the DB node in the Services tab? There is probably no memory leak there, right? Do you have some idea how this could be fixed for Hibernate? Thanks.
Comment 9 Sergey Petrov 2011-03-02 17:47:30 UTC
waiver candidate - it's likely exists since hibernate module implementation and one report so far, I can see the leak but don't see parmgen after dozen of generations, it's likely isn't reproducible in most usual usecases/usual workflow with 1-2 generation in a project.
Comment 10 Petr Jiricka 2011-03-08 10:22:45 UTC
Waiver approved.
Comment 11 Sergey Petrov 2011-04-22 14:50:35 UTC
as I have no good solution for security issues, and db module can't be used here (as projects classpath have to be used in hql), decide to cache classloader with weakreference, it should almost resolve the issue (likely one classloader per project instead of one on each hql execution/query modification in the same project). In general the issue may never appear in usual workflow now but I'm not sure it can be marked as fixed also. But may be can be downgraded to p3, 
Petr, what's  your opinion?
(in some specific cases like use execute hql in 30-40 projects of execute, "modify project claaspath -> execute -> modify -> execute ->" sequence for 30-40 times it may appear, yet I wasn't able to reproduce it even before with reasonable time to reproduce)

http://hg.netbeans.org/web-main/rev/fa7571b6b6f2 cache classloader
and also sync classpath for hql parsing and hql execution.
Comment 12 Quality Engineering 2011-04-23 04:37:17 UTC
Integrated into 'main-golden', will be available in build *201104230000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/fa7571b6b6f2
User: Sergey B. Petrov <sj-nb@netbeans.org>
Log: fix #194449 cache classloader for a project with WeakReference, sync classpath for hql parse and execution
Comment 13 Sergey Petrov 2011-04-29 12:29:21 UTC
>p3 for now before more evaluation or more reports
Comment 14 Sergey Petrov 2011-11-01 12:25:38 UTC
it seems no more permgen reports so far, should be quite rare issue but still good to reevaluate later.
Comment 15 Sergey Petrov 2012-05-17 12:54:57 UTC
I see no new reports after "part fix", may be extremly rare now or can be considered as fixed.