Bug 4847 - major bayes_seen fh leak
Summary: major bayes_seen fh leak
Status: RESOLVED WORKSFORME
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Learner (show other bugs)
Version: 3.1.1
Hardware: All other
: P3 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-03-30 19:38 UTC by Nick Alcock
Modified: 2006-08-02 11:55 UTC (History)
0 users


Attachment Type Modified Status Actions Submitter/CLA Status
spamassassin -D -r . output text/plain None Nick Alcock [HasCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Alcock 2006-03-30 19:38:47 UTC 
I've just noticed this when doing a `spamassassin -r' of an entire directory:

nix@loki 38 /home/nix% ps -fC spamassassin
USER       PID  PPID  C STIME TTY          TIME CMD
nix      23188  5468 21 19:31 pts/1    00:00:12 /usr/bin/perl -T -w
/usr/bin/spamassassin -r .

nix@loki 40 /proc/23188/fd% ls -l
total 9
lrwx------ 1 nix users 64 Mar 30 19:32 0 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:32 1 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:32 2 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:32 3 -> /home/nix/.spamassassin/bayes_seen
lr-x------ 1 nix users 64 Mar 30 19:32 4 -> pipe:[654428]
lrwx------ 1 nix users 64 Mar 30 19:32 5 -> socket:[654246]
lrwx------ 1 nix users 64 Mar 30 19:32 6 -> /home/nix/.spamassassin/bayes_seen
l-wx------ 1 nix users 64 Mar 30 19:32 7 -> /home/nix/.razor/razor-agent.log
lrwx------ 1 nix users 64 Mar 30 19:32 8 -> socket:[654893]

[wait a moment]

nix@loki 50 /proc/23188/fd% ls -l
total 25
lrwx------ 1 nix users 64 Mar 30 19:32 0 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:32 1 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:34 10 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:34 11 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:34 12 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:34 13 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 14 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 15 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 16 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 17 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 18 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 19 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:32 2 -> /dev/pts/1
lrwx------ 1 nix users 64 Mar 30 19:38 20 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 21 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 22 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 23 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:38 24 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:32 3 -> /home/nix/.spamassassin/bayes_seen
lr-x------ 1 nix users 64 Mar 30 19:32 4 -> pipe:[657079]
lrwx------ 1 nix users 64 Mar 30 19:32 5 -> socket:[654246]
lrwx------ 1 nix users 64 Mar 30 19:32 6 -> /home/nix/.spamassassin/bayes_seen
l-wx------ 1 nix users 64 Mar 30 19:32 7 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:32 8 -> /home/nix/.spamassassin/bayes_seen
lrwx------ 1 nix users 64 Mar 30 19:34 9 -> /home/nix/.spamassassin/bayes_seen


I ran out of file handles trying to report a three-day round of five thousand
spams...

It seems like we leak one every three or four messages.
Comment 1 Nick Alcock 2006-03-30 19:48:59 UTC 
I'll send some -D info as soon as this run is over. It might be a DB_File bug, I
suppose (I'm still on 4.2.52), but then again it might be a missing untie_db()
call (although looking at the code I can't easily see how).
Comment 2 Nick Alcock 2006-03-30 20:06:50 UTC 
Created attachment 3437 [details]
spamassassin -D -r . output

Well, here's a bit of debugging. Tie and untie look precisely matched: db-4.2
bug, I fear :(

(I knew I had to upgrade, but the damn constant binary incompatibility of libdb
made me put it off for a long time.)

bayes_seen was open four times by the end of this (<1min long) run.
Comment 3 Theo Van Dinter 2006-03-30 21:30:45 UTC 
(In reply to comment #2)
> Well, here's a bit of debugging. Tie and untie look precisely matched: db-4.2
> bug, I fear :(

I know that I debugged this type of thing a while ago, but I can't find a mail or ticket with the info.  The 
problem I found before was that we called untie appropriately, but there was an error within the library 
function which caused the close to not happen.
Comment 4 Theo Van Dinter 2006-08-02 02:04:22 UTC 
hearing nothing else, closing as wfm.
Comment 5 Nick Alcock 2006-08-02 06:59:06 UTC 
Oops, sorry: I finally upgraded libdb to 4.4.20 last week.

The fault persists, alas.

I'll leave it at wfm until I can debug this myself: I shouldn't put the load on
people who can't reproduce it.
Comment 6 Theo Van Dinter 2006-08-02 14:40:20 UTC 
(In reply to comment #5)
> Oops, sorry: I finally upgraded libdb to 4.4.20 last week.
> 
> The fault persists, alas.

Did you also upgrade DB_File?
Comment 7 Nick Alcock 2006-08-02 18:55:02 UTC 
Yes, currently at 1.814 (latest on CPAN). It's linked against the new libdb:

nix@hades 120 /home/nix% ldd
/usr/lib/perl5/5.8.8/i686-linux-thread-multi-ld/auto/DB_File/DB_File.so
        linux-gate.so.1 =>  (0xffffe000)
        libdb-4.4.so => /usr/lib/libdb-4.4.so (0xb7eac000)
        libc.so.6 => /lib/libc.so.6 (0xb7d6f000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xb7d5c000)
        /lib/ld-linux.so.2 (0x80000000)