Bug 5452 - Many circular references
Summary: Many circular references
Status: NEW
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Libraries (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: Other other
: P5 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-05-08 12:37 UTC by Dianne Skoll
Modified: 2010-07-13 03:44 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Devel::Leak::Object memory leaks on Debian5 text/plain None Daniel Lemke [HasCLA]
Devel::Leak::Object memory leaks on Win32 text/plain None Daniel Lemke [HasCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Dianne Skoll 2007-05-08 12:37:53 UTC 
The SpamAssassin libraries create many, many circular references.  Unless you
are very careful, this could lead to a memory leak.

Example:  Dumping a Mail::SpamAssassin object after a SpamAssassin run reveals
about 233 lines of back-references.

The circular references are used for two purposes as far as I can see:

1) a "main" backreference back to the main SpamAssassin object so plugins can
reach back into it.

2) "performance" backreferences to save typing and time so you don't need to
dereference 4 or 5 hashes and arrays to walk the data structure.

(1) could be solved by passing the main SpamAssassin object around as an
argument to methods and functions that need it.  (2) might be solvable by
limiting the scope of the performance backreferences to "my" variables.

Fixing this will be an awful lot of work.  Maybe we should wait for Perl to have
a proper mark/sweep GC engine. :-)
Comment 1 Matt Sergeant 2007-05-09 08:02:55 UTC 
There's no reason those links couldn't be weaken()'d - we now require perl 5.6 IIRC?
Comment 2 Justin Mason 2007-05-09 08:16:57 UTC 
yep, perl 5.6.1 is our current minimum.  I didn't realise we could safely weaken
refs with that - can we?
Comment 3 Matt Sergeant 2007-05-09 08:29:55 UTC 
http://search.cpan.org/~lukka/WeakRef-0.01/WeakRef.pm

Says it arrived in 5.005_55 (i.e. perl 5.6.0).
Comment 4 Justin Mason 2007-05-09 08:41:19 UTC 
http://www.twoshortplanks.com/modulecorelist/index.cgi?module=WeakRef&version=&submit=Lookup
: 'Module WeakRef was never distributed with perl.'

:(
Comment 5 Dave O'Neill 2007-05-09 09:39:44 UTC 
(In reply to comment #4)
>
http://www.twoshortplanks.com/modulecorelist/index.cgi?module=WeakRef&version=&submit=Lookup
> : 'Module WeakRef was never distributed with perl.'
> 
> :(

You don't want WeakRef, you want Scalar::Util.

It didn't hit core perl until 5.8, though.
Comment 6 Daniel Lemke 2010-07-05 09:17:57 UTC 
(In reply to comment #5)
> You don't want WeakRef, you want Scalar::Util.
> 
> It didn't hit core perl until 5.8, though.


Is Perl 5.6.1 still minimum? If possible, I'd like to have the weaken() calls in a future release. Circular references are causing a huge memory leak for spamd that makes it necessary to restart our daemon every hour or so.

Daniel
Comment 7 Daniel Lemke 2010-07-13 03:43:50 UTC 
Created attachment 4787 [details]
Devel::Leak::Object memory leaks on Debian5
Comment 8 Daniel Lemke 2010-07-13 03:44:53 UTC 
Created attachment 4788 [details]
Devel::Leak::Object memory leaks on Win32

Hmm no comments on this? :(

I did some little testing with Devel::Leak on a Win32 and a Debian5 system. Both of them are showing similar issues in terms of memory leak (reports attached).

Looks like the problem is more serious for Windows, so I had a look on the differences:

Sources of leaks:
	Math::BigInt                             1
	NetAddr::IP                              8
	Win32::API                               14
	Win32::Registry                          7

These are not occuring on unix boxes (what a suprise for the last ones) so it may be the cause of the huge leaks.

I guess this is more of a problem of Win32 perl modules, than of SpamAssassin, is that right?

Daniel