Bug 5402 - SpamCop plugin occasionally timeouts reporting spam
Summary: SpamCop plugin occasionally timeouts reporting spam
Status: NEW
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Plugins (show other bugs)
Version: 3.1.8
Hardware: Other Linux
: P5 enhancement
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
Depends on:
Reported: 2007-04-03 04:59 UTC by Giampaolo Tomassoni
Modified: 2008-08-01 14:34 UTC (History)
1 user (show)

Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Giampaolo Tomassoni 2007-04-03 04:59:26 UTC
The Mail::SpamAssassin::Plugin::SpamCop module sometimes timeouts reporting 
spam to SpamCop.

This seems to happe when the SpamCop servers are enough busy to let the plugin 
SMTP connection to timeout.

I see that rising the timeout value from the default (10 seconds) to, say, 30 
seconds does solve the problem.

I would suggest to keep the current default, while allowing to specify a 
timeout value by a config key (say, spamcop_report_timeout ?). Thereby, people 
experiencing this problem and willing to do a better effort in reporting to 
spamcop may choose to raise the default timeout value through that config key.
Comment 1 Steven Chamberlain 2008-08-01 13:46:04 UTC

(btw: I feel this issue may be a 'bug', rather than a request for enhancement as it's currently flagged)

I shall provide a little more detail.  If reporting to SpamCop with 'spamassassin --report' times out, the following warnings are shown:

[11530] warn: reporter: SpamCop report to vmx1.spamcop.net failed: Net::SMTP error
[11530] warn: reporter: SpamCop report to vmx2.spamcop.net failed: Net::SMTP error
[11530] info: reporter: could not report spam to SpamCop
[11530] warn: reporter: no reporting methods available, so couldn't report
[11530] warn: spamassassin: warning, unable to report message
[11530] warn: spamassassin: for more information, re-run with -D option to see debug output

I suffer this problem regularly, and have only just realised what was causing it.  The above errors can also be found in mailing list archives, and it seems that people have been affected by this but never discovered the cause:
in 2002, http://mail-archives.apache.org/mod_mbox/spamassassin-users/200503.mbox/%3C9a80add1542b60f0866bb414ef36439f@comcast.net%3E
in 2007, http://groups.google.com/group/spamcop.help/browse_thread/thread/f7e2eed18e753ce5

Exactly what happens is, after the server's 220 greeting, the reporter sends the EHLO and waits for a 250 response before proceeding.  If this doesn't happen within 10 seconds, the reporter times out, closing the connection and showing the vague 'Net::SMTP error'.  Both of the two available MXes are tried, and if both attempts time out, the reporter aborts the SpamCop submission and displays the warnings above (but still returns a zero 'success' error code).

The 10-second timeout strikes me as unusually low.  RFC 2821 section, 'Timeouts', suggests minimum timeouts of 5 minutes for the MAIL and RCPT commands.  Whilst it does not suggest a timeout for the EHLO command, the same period of time seems appropriate for EHLO as for MAIL and RCPT.

Note that the RFC specifies shorter timeouts (2 and 3 minutes) or longer timeouts (10 minutes) for other SMTP commands, and I don't believe Net::SMTP offers that flexibility.

Therefore I believe this justifies raising the default from 10 seconds to at least 5 minutes, and preferably 10 minutes (until such a time as Net::SMTP allows timeouts to be specified differently for each SMTP command).  However, I'm aware that in some situations people may consider these periods excessive, alas, the RFC also states:
  Timeouts [in SMTP clients] SHOULD be easily reconfigurable, preferably without recompiling the SMTP code.

Whilst Net::SMTP is effectively the 'SMTP client' in this situation, and its timeout is easily configurable, I think the reporter is wrong to specify a hard-coded timeout.

In summary, I propose three changes:

1. make sure that an SMTP timeout results in a more verbose warning, at least when using the '-D' option
2. add a configuration option for the SMTP command timeout
3. raise the default SMTP command timeout from 10 seconds to 10 minutes

I don't feel competent enough to create a patch implementing changes 1 and 2, but 3 is so trivial that applying a patch would be actually require more effort than editing the plugin file directly.  In the SpamAssassin v3.2.5 distribution the timeout is specified in file lib/Mail/SpamAssassin/Plugin/SpamCop.pm at line 271.

Comment 2 Giampaolo Tomassoni 2008-08-01 14:04:05 UTC
Well, I don't know if this can be thought as related, but another way to circumvent this (yet adding a very useful feature) would be to allow a way to specify a proxy smtp server to use in order to send reports to SC.

Something like:

	spamcop_report_smtp  SMTP_SRVR_NAME_OR_ADDRESS[:PORT]

would really be appreciated. This way 'spamassassin -r' would definitely take less real time to complete and reporting would be much more reliable.

Also, the PORT option would be necessary for the ones (like me) who run an smtp-outgoing service on the very server in which they run an amavisd-monitored ingoing service: spam messages reported to SC should not pass through spamassasin again...
Comment 3 Steven Chamberlain 2008-08-01 14:34:46 UTC
That sounds like a nice idea to me -- a configuration option for an SMTP relay to use for sending the report, rather than connecting directly to the SpamCop MXes.  That feature has already been requested as bug #4684, and I think it may be a better solution for this bug also.

But I notice the reports are submitted to the SpamCop MXes on port 587 instead of the standard port.  I imagine that is to avoid transparent, filtering SMTP relays which some ISPs seem to use.  Hopefully it's still possible for people to submit the report to SpamCop on the standard SMTP port if they know their ISP does not do this kind of filtering.  And otherwise, their SMTP relay would have to be configured to deliver SpamCop reports on port 587 (assuming that port isn't filtered too).

In my situation I could simply put the report into the system's normal mail queue and I think it would be delivered.  The reporter would 'complete' almost immediately, whilst the relay takes care of delivery attempts.  Although some people might want to be able to specify an SMTP relay for situations where they filter their outgoing mail, or need to deliver to port 587 as I mentioned above.