Bug 5586

Summary: RDNS_NONE has false positives if the MTA doesn't put the hostname in the Received header
Product: Spamassassin Reporter: John Rudd <john>
Component: RulesAssignee: SpamAssassin Developer Mailing List <dev>
Status: RESOLVED FIXED    
Severity: normal    
Priority: P5    
Version: 3.2.1   
Target Milestone: 3.2.4   
Hardware: Sun   
OS: Solaris   
Whiteboard:

Description John Rudd 2007-08-05 18:08:58 UTC
I sent this to the mailing list, but was asked by Justin to submit it as a bug.  Doing so now.

After I sent my message, Matthew Yette posted a reply to my message saying that the bug also applies 
to qmail 1.03 as well.  He didn't give an example mail header.


The following received header gets RDNS_NONE:

Received: from [69.12.154.165] (HELO mail.rudd.cc)
  by some.internal.test.host (CommuniGate Pro SMTP 5.1.11)
  with ESMTP id 60056 for jrudd@some.internal.test.domain; Tue, 31 Jul 2007 14:05:04 -0700



This is an error in the RDNS_NONE logic.  The host in question DOES have RDNS.  The issue is that 
CommuniGate Pro doesn't put the RDNS information into the Received header (and the place where it 
puts the HELO information might instead have the authenticated user, if SMTP-AUTH was used).


My recommendation is that the logic for RDNS_NONE should be:


if (Received =~ /\(CommuniGate Pro SMTP /) {
   if (DNS checks are enabled) {
      do the DNS check on the IP address, and then decide if it has RDNS
      }
   else {
      don't trigger RDNS_NONE (because you can't know without the DNS check)
      }
   }
else { # not a CommuniGate Pro MTA
   do existing RDNS_NONE logic
   }
Comment 1 John Rudd 2007-08-05 18:14:31 UTC
> if (Received =~ /\(CommuniGate Pro SMTP /) {


Thinking more about it, the above line probably should omit the protocol (SMTP), because the same basic 
problem will happen if it's from webmail, etc.  So, a better expression would probably be:

if ( Received =~ /by \S+ \(CommuniGate Pro \S+ [0-9\.c]+\)/ ) {
Comment 2 John Rudd 2007-08-05 18:17:54 UTC
(In reply to comment #1)

> 
> if ( Received =~ /by \S+ \(CommuniGate Pro \S+ [0-9\.c]+\)/ ) {
> 

Sorry, just realized that doesn't work if it's a basic/free/community license (it puts _community_ after 
the version number, and before the closing parenthesis).  This is probably better:

 if ( Received =~ /by \S+ \(CommuniGate Pro/ ) {


Comment 3 Matthew Yette 2007-08-06 05:27:50 UTC
My apologies for not including a sample header. Here is one:

Return-Path: <myette@mvnhealth.com>
Delivered-To: matt@integrityhosting.org
Received: (qmail 20591 invoked by uid 511); 6 Aug 2007 12:24:23 -0000
Received: from 64.9.116.126 by mail.integrityhosting.org (envelope-from
<myette@mvnhealth.com>, uid 509) with qmail-scanner-1.25st
     (clamdscan: 0.91.1/3302. spamassassin: 3.2.2. perlscan: 1.25st.
     Clear:RC:0(64.9.116.126):SA:0(-2.5/4.0):.
     Processed in 2.197283 secs); 06 Aug 2007 12:24:23 -0000
X-Spam-Status: No, hits=-2.5 required=4.0
X-Spam-Report: SA TESTS
     0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
     -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
     [score: 0.0000]
     0.0 HTML_MESSAGE BODY: HTML included in message
     0.0 AWL AWL: From: address is in the auto white-list
Received: from unknown (HELO gwia1-lx.mvnhealth.com) (64.9.116.126)
     by 0 with SMTP; 6 Aug 2007 12:24:20 -0000
Received: from MVN-INT3-MTA by gwia1-lx.mvnhealth.com
     with Novell_GroupWise; Mon, 06 Aug 2007 08:24:19 -0400
Message-Id: <46B6DAA5.4335.00D3.0@mvnhealth.com>
X-Mailer: Novell GroupWise Internet Agent 7.0.1
Date: Mon, 06 Aug 2007 08:24:05 -0400
From: "Matthew Yette" <myette@mvnhealth.com>
To: <matt@integrityhosting.org>
Subject: test2
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="=__Part7E5982F5.0__="
Comment 4 Justin Mason 2007-09-12 05:51:41 UTC
added RDNS_NONE_BUG_5586 for testing, to see what effect this change has:

: jm 49...; svn commit -m "bug 5586: add a test rule to avoid RDNS_NONE false
positives" rulesrc/sandbox/jm/20_bug5586.cf
Adding         rulesrc/sandbox/jm/20_bug5586.cf
Transmitting file data .
Committed revision 574928.
Comment 5 Justin Mason 2007-09-12 05:52:38 UTC
(In reply to comment #3)
> My apologies for not including a sample header. Here is one:

> Received: from unknown (HELO gwia1-lx.mvnhealth.com) (64.9.116.126)
>      by 0 with SMTP; 6 Aug 2007 12:24:20 -0000

"by 0"? 

in the qmail case, isn't it possible to simply enable rDNS lookups in the qmail
config?
Comment 6 Justin Mason 2007-09-13 03:57:58 UTC
(In reply to comment #4)
> added RDNS_NONE_BUG_5586 for testing, to see what effect this change has:
> 
> : jm 49...; svn commit -m "bug 5586: add a test rule to avoid RDNS_NONE false
> positives" rulesrc/sandbox/jm/20_bug5586.cf
> Adding         rulesrc/sandbox/jm/20_bug5586.cf
> Transmitting file data .
> Committed revision 574928.

looks ok so far:
http://ruleqa.spamassassin.org/?daterev=20070913-r575223-n&rule=%2FRDNS_NONE&srcpath=&g=Change




Comment 7 Justin Mason 2007-09-13 06:52:19 UTC
looks great! 

http://ruleqa.spamassassin.org/?daterev=20070913-r575223-n&rule=%2FRDNS_NONE&srcpath=&g=Change
 
0.00000 	42.6479  16509 of 38710 messages  	 2.7170  736 of 27089 messages  	
0.940 	 0.69 	 0.00 	T_RDNS_NONE_BUG_5586 	 	
0.00000 	42.6582 16513 of 38710 messages 	2.7465 744 of 27089 messages 
0.940 	0.69 	0.10 	RDNS_NONE 		


so, +1 to replacing it.
Comment 8 Justin Mason 2007-09-24 05:20:29 UTC
applied to trunk:

: jm 22...; svn commit -m "bug 5586: RDNS_NONE has false positives if the MTA
doesn't put the hostname in the Received header, like Communigate Pro. add an
exception for this" rules/20_dynrdns.cf rulesrc/sandbox/jm/
Deleting       rulesrc/sandbox/jm/20_bug5586.cf
Sending        rules/20_dynrdns.cf
Transmitting file data .
Committed revision 578777.

applied to 3.2.x:

: jm 50...; svn commit -m "bug 5586: RDNS_NONE has false positives if the MTA
doesn't put the hostname in the Received header, like Communigate Pro. add an
exception for this" rules/20_dynrdns.cf
Sending        rules/20_dynrdns.cf
Transmitting file data .
Committed revision 578778.

applied to 3.2.x rule updates:

: jm 37...; svn commit -m "bug 5586: RDNS_NONE has false positives if the MTA
doesn't put the hostname in the Received header, like Communigate Pro. add an
exception for this" 20_dynrdns.cf
Sending        20_dynrdns.cf
Transmitting file data .
Committed revision 578781.

Comment 9 Sidney Markowitz 2007-12-15 19:44:59 UTC
There was a typo in the 3.2 checkins. The header rule was left named RDNS_NONE
instead of __RDNS_NONE.

$ svn ci -m "bug 5586: fix typo in the fix that was checked in to branch 3.2"
rules/20_dynrdns.cf 
Sending        rules/20_dynrdns.cf
Transmitting file data .
Committed revision 604549.
$ cd ../../rules/branches/3.2/
$ svn ci -m "bug 5586: fix typo in the fix that was checked in to branch 3.2
rules update" 20_dynrdns.cf 
Sending        20_dynrdns.cf
Transmitting file data .
Committed revision 604550.