Created attachment 2602 [details]
patch against 3.0

I've been running this for about a day now... works pretty nice.  I should have
wrote it sooner.

Adds --max-load-average option to spamd.  Once the set load average threshold
is exceeded spamd (almost) immediately closes the connection on the spamc
client, causing spamc to try another spamd server (if the spamc -d flag is
used) or return the unscanned message.	The child's connection counter is not
incremented so that killing & respawning the child doesn't add to the system's
load.

If there's interest, I'll edit the patch to work against trunk too.


Daryl

Oops... -d doesn't fall back to the next host listed, it just returns the
message unscanned.

Martin, Daryl, would a viable alternative, for some sites, be: 
If system load gets very high, then freeze mail processing. Don't drop emails;
don't ignore them. Allow them to queue up, and when system load drops to
reasonable levels, process those emails. 

The mechanism would have to be sturdy and survive reboot (ie: system load goes
through the roof, and admin "fixes" the problem by rebooting the system -- the
email system should pick up where it left off). 

Subject: Re:  RFE: stop spam detection at maximum load average

How about this as a maintainable way?

 - We add a plugin API where various bits of code can query the system
   load and available memory.

 - The API is farmed out to OS-specific plugins to implement those
   methods.

 - Plugins that consume lots of resources can call that API to determine
   the system load and available memory and act accordingly.

For example, TextCat could call the API and determine whether or not to
run.  The same for network tests (memory more than CPU), Bayes, AWL, and
even foreign language body rules (CPU)!

This API is something we can do without relying on any type of message
queueing.

Bob, stopping mail receiving is not an option for a site thaz get 15.000 to
20.000 mails per day (not counting the ones rejected by RBL).

Daniel, your proposal sounds interesting. Better to have some weak defences than
none.

(In reply to comment #4)
> Subject: Re:  RFE: stop spam detection at maximum load average
> 
>  - Plugins that consume lots of resources can call that API to determine
>    the system load and available memory and act accordingly.
> 
The plugin that checks the system load, and decides not to run should then
report this is some way. Otherwise, there could be a spate of bugs like "X
doesn't work sometimes" etc.

This can now be done, including what Daniel outlined, with a custom Check
plugin.  Since it's highly dependent on the underlying OS it's not going to
happen in time for 3.2 though.  Kicking to 3.3.

Btw, a widespread usage of such feature would encourage spammers
to shift their spamming strategy to send their junk in bursts
to target hosts, instead of more randomly spreading across
recipients from various domains. It would be easy enough to do
so, and everone would then suffer. IMO it would be a misfeature.

moving most remaining 3.3.0 bugs to 3.3.1 milestone

reassigning, too

Retargeting to: Undefined

I think that Mail::SpamAssassin::Plugin::ResourceLimits could be used for this purpose nowadays.

Mail::SpamAssassin::Plugin::ResourceLimits can be used starting from 3.4.2,
older versions are not supported or does not make sense to add compatibility glue.