Bug 241 - SA-2.20 (official) dies with "-D" disabled
Summary: SA-2.20 (official) dies with "-D" disabled
Status: RESOLVED INVALID
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: spamc/spamd (show other bugs)
Version: unspecified
Hardware: PC Linux
: P2 major
Target Milestone: ---
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-04-21 15:20 UTC by Jason Haar
Modified: 2002-04-21 15:19 UTC (History)
0 users


Attachment Type Modified Status Actions Submitter/CLA Status
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Haar 2002-04-21 15:20:57 UTC 
I just installed SA-2.20, and threw a few messages I had through it to test that it  
worked.  I run it as: 
 
spamd -i 0.0.0.0 -A 127.0.0.1,155.63.248.198 -F 1 -L -x -u spamc -m 10  
 
On some messages, it's fine, on others (looks like messages with attachments) spamd 
logs via syslog the connection from spamc, but no action. And the message has no 
headers added. This is irrespective of whether or not the message is spam or not. 
 
If I add "-D" to the end of that, and run it manually, then all messages are marked 
correctly. It looks like "-D" is needed for some output senarios?
Comment 1 Jason Haar 2002-04-21 15:25:41 UTC 
  
Gah - me bad. It's a memory problem. I've been running SA-2.20 from CVS for some time, 
but it looks like there's large memory requirments when dealing with mail with 
attachments? I run spamd under daemontools with a 10Mb RAM limit. Running it with "-D" 
was a red-hearing - it was running it with no resource limit that fixed it. 
 
Should SA use so much RAM? Shouldn't it be using a moving-window to minimize memory 
usage?
Comment 2 Craig Hughes 2002-04-21 17:32:07 UTC 
I suspect what you're seeing is the spamc default of setting "-s 250000" -- in other words, messages longer than 250k won't get processed by spamd under default settings.  Messages with attachments tend to be the ones that are >250k.  Because of a curiosity in the way the message is processed, spamc will in fact open a connection to spamd, but then once it realizes the message is >250k, it'll just close the connection again, hence spamd logs a connection being opened, but never mentions receiving an actual message.  Go ahead and re-open if it turns out this isn't the answer.
Comment 3 Jason Haar 2002-04-21 17:49:11 UTC 
 No - it's not a "-s" issue. All messages are < 100K, and they're processed fine when I 
upped the softlimit on the amount of RAM spamd could use. 
 
It's simply that something in SA must be creating a large array or variable containing 
the content of the message or the like. I had a softlimit of 10M set for spamd - such 
messages needed around 12Mb RAM... 
 
There must have been new tests added to SA that have pushed it's memory requirements up 
by a couple of megs. 
 
BTW, if I had 3 such messages coming in simultaneously, would that be 12x3 MB RAM or 
just 12Mb + a bit? This is probably an issue that should be documented as if an ISP is 
wanting to process >20 messages at a time via spamd, and a proportion of them are 
swallowing largish chunks of RAM, then the system may become unstable. 
 
I'm recommending everyone run spamd under softlimits/ulimits just as Qmail runs 
qmail-smtpd: it stops unstable conditions from occurring.
Comment 4 Craig Hughes 2002-04-21 23:19:32 UTC 
You should have mentioned you limited SA with a softlimit.  It's not crashing on it's own, it's being killed by the OS!  In answer to the question on whether it will be 12 Mb per message or 12 MB + a little bit, it'll depend on your OS.  Most modern OSes will be 12 + a bit.  Though each process will show up in the process table as having 12 MB, most of the memory will be shared through copy-on-write stuff.