Bug 2634 - rule: based on SPF "best guess"
Summary: rule: based on SPF "best guess"
Status: RESOLVED WONTFIX
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (Eval Tests) (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: Other other
: P5 enhancement
Target Milestone: 3.1.0
Assignee: Justin Mason
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-10-27 11:22 UTC by Justin Mason
Modified: 2005-01-19 05:33 UTC (History)
1 user (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Justin Mason 2003-10-27 11:22:31 UTC
This is an interesting technique.   IMO, it may be better to do it locally
inside SpamAssassin as its own rule, since I reckon *real* SPF records
will probably have higher accuracy, and therefore higher S/O.  The
heuristic is:

  if the client IP is in the same class C as the domain's A record, pass.
  if the client IP is in the same class C as the domain's MX servers, pass.
  if the client IP's PTR record matches the domain name, pass.

One to try out for 2.70.




 Date:    Sun, 26 Oct 2003 20:02:52 -0500
 From:    Meng Weng Wong <mengwong@dumbo.pobox.com>
 Subject: Mail::SPF::Query has a best_guess method

...
So I've got an even bigger plus for you: Mail::SPF::Query now has a
"best_guess" method, which basically does this:

  my $BEST_GUESS_MECHS = "a/24 mx/24 ptr ptrhs:wl.trusted-forwarder.org";

So:
  if the client IP is in the same class C as the domain's A record, pass.
  if the client IP is in the same class C as the domain's MX servers, pass.
  if the client IP's PTR record matches the domain name, pass.
  if the client IP's PTR domain name is found in wl.trusted-forwarder.org, pass
.

R. Scott Perry of Declude JunkMail did some testing on his spambase:

    Here are the results.  This includes all spam to our spamtraps received in
    an entire one week period ending this morning around 8AM.

    Total spams:                    33,809
    Total failing "a mx ptr":               31,270  [92.5%]
    Total failing "ptr":            32,345  [95.7%]
    Total failing "mx":             31,556  [93.3%]
    Total failing "a":              31,420  [92.9%]

    So if all domains had an SPF entry of "a mx ptr", 92.5% of the spam we
    received over the past week would have been caught.

    The results for the ham were quite interesting:

    Total ham:                      1860
    Total failing "a mx ptr":                504    [27.1%]

    This looks pretty bad -- 27.1% of ham E-mail failing the "a mx ptr" test.

    However, after I looked at the ham that failed the test, I found that most
    of them have an IP in the same Class C range as an IP in the MX/A
    records.  Often, mailservers are set up to use many IPs (usually on the
    same Class C range), and they may "pick" an IP that isn't in the MX record,
    even though it is the same mailserver.  Or the mail comes out through a
    gateway server, typically also in the same Class C range.

    Those numbers go down to 142 E-mails (7.6%) failing "a mx ptr" simply by
    changing the MX and A tests to pass if the IP is in the same Class C range,
    rather than looking for an exact match.  The numbers for spam failing "a mx
    ptr" only go down from 92.5% to 87.7% with this change.
Comment 1 Justin Mason 2003-10-27 11:23:38 UTC
grabbing! ;)
Comment 2 Robert J. Accettura 2004-01-09 06:19:44 UTC
SPF is now implemented by AOL (first mega-isp to do so):
http://slashdot.org/article.pl?sid=04/01/09/0435234&mode=thread&tid=111&tid=120&tid=126&tid=187

Would be nice if SA uses this, as it's a great way to fight spam.
http://spf.pobox.com/
Comment 3 Justin Mason 2004-01-09 11:28:58 UTC
Robert, SA 2.70 contains SPF support already.
Comment 4 Robert J. Accettura 2004-01-09 12:01:10 UTC
Justin, thanks.  Wasn't aware.  I've been focusing on 2.6x, and SAProxy lately.
 I'm a bit out of the loop as far as 2.70.  

Trying to implement SPF myself later today on my domains.
Comment 5 Justin Mason 2004-02-18 21:03:02 UTC
taking
Comment 6 Theo Van Dinter 2004-03-11 13:44:53 UTC
ok, so is this ticket done since there are SPF rules in 3.0?
Comment 7 Justin Mason 2004-03-14 21:03:53 UTC
nah, it's not -- it didn't give good results *at all* in my earlier testing
(see rule T_ALL_RELAYS_NEAR_MXES).  Punting to 3.1.0 for further investigation,
if any.
Comment 8 Justin Mason 2004-03-23 20:12:09 UTC
removing from blocker
Comment 9 Daniel Quinlan 2004-08-27 17:17:43 UTC
more accuracy and performance bugs going to 3.1.0 milestone
Comment 10 Daniel Quinlan 2005-01-19 14:20:16 UTC
What about using "best guess" as a whitelisting technique for ham?  See
which domains always work with a reasonable best guess and then we can do
SPF best-guess-based whitelisting of those domains.

I have heard someone get good results with a best guess record of:

 "v=spf1 ptr ?all"

We could also test passes using other best guess record types.
Comment 11 Justin Mason 2005-01-19 14:33:37 UTC
it might work on a domain-by-domain basis -- but AFAICS your suggested string is
basically checking the reverse DNS, right?  we already have (or had) a rule for
that, and it doesn't work great.