Bug 2497 - Dynamic/Residential IP Range checking is too aggressive
Summary: Dynamic/Residential IP Range checking is too aggressive
Status: RESOLVED WORKSFORME
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 2.60
Hardware: PC Linux
: P5 normal
Target Milestone: 2.70
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-09-23 15:40 UTC by David L. Crow
Modified: 2003-10-03 05:07 UTC (History)
2 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
rfc822 format message that shows the issue text/rfc822 None David L. Crow [NoCLA]
stderr from "spamassassin -D" on previously attached message. text/plain None David L. Crow [NoCLA]
DNSBLs test dialup lists one hop too far message/rfc822 None Dale Blount [NoCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description David L. Crow 2003-09-23 15:40:48 UTC
I received an email with the following Received headers (full message to be
created as an attachment):

    Received: from ms-smtp-02.texas.rr.com (ms-smtp-02.texas.rr.com
      [24.93.36.230]) by waterloo.OrangeBlood.org (8.12.8/8.12.8)
      with ESMTP id h8NDa2UW031261 for <crow@OrangeBlood.org>;
      Tue, 23 Sep 2003 08:36:02 -0500
    Received: from 448xc01 (cs2439-220.austin.rr.com [24.243.9.220])
      by ms-smtp-02.texas.rr.com (8.12.5/8.12.2) with SMTP id
      h8NDZk8V029596; Tue, 23 Sep 2003 08:35:47 -0500 (CDT)

And the scoring system provided:

    3.5 RCVD_IN_NJABL_DIALUP   RBL: NJABL: dialup sender did non-local
                               SMTP [24.243.9.220 listed in
                               dnsbl.njabl.org]
    2.6 RCVD_IN_DYNABLOCK      RBL: Sent directly from dynamic IP
                               address [Dynamic/Residential IP range
                               listed by] [easynet.nl DynaBlock -
                             <http://dynablock.easynet.nl/errors.html>]
    0.1 RCVD_IN_NJABL          RBL: Received via a relay in
                               dnsbl.njabl.org [24.243.9.220 listed in
                               dnsbl.njabl.org]

The mail was sent from a "dynamic host" directly to the appropriate ISP SMTP
server using "Outlook Express 6.00.2800.1158" and then delivered to my host. 
Should this message really have been marked as coming from a dynamic host? 
Can/should the first Received: header be ignored?
Comment 1 David L. Crow 2003-09-23 15:55:13 UTC
Created attachment 1420 [details]
rfc822 format message that shows the issue

Removing much of the personal information seems to have gotten the score down
below a 5, so some of the other matching rules must have generated negative
numbers becuase just the 3 residential ip rules add to larger than 5.
Comment 2 David L. Crow 2003-09-23 16:06:22 UTC
~/.spamassassin/user_prefs containts

  use_terse_report 1
  report_safe 2
  required_hits     5

and a bunch of whitelist_from and whitelist_to entries.

/etc/mail/spamassassin/local.cf only contains comment lines.
Comment 3 David L. Crow 2003-09-23 16:08:12 UTC
Sorry I didn't get all of this in on one trye.

Here is the report from spamassassin -t on the "filtered" message:

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.1 HTML_60_70             BODY: Message is 60% to 70% HTML
-1.5 BAYES_01               BODY: Bayesian spam probability is 1 to 10%
                            [score: 0.0201]
 0.1 HTML_MESSAGE           BODY: HTML included in message
 2.6 SUSPICIOUS_RECIPS      Similar addresses in recipient list
 3.5 RCVD_IN_NJABL_DIALUP   RBL: NJABL: dialup sender did non-local SMTP
                            [24.243.9.220 listed in dnsbl.njabl.org]
 2.6 RCVD_IN_DYNABLOCK      RBL: Sent directly from dynamic IP address
                            [Dynamic/Residential IP range listed by]
            [easynet.nl DynaBlock - <http://dynablock.easynet.nl/errors.html>]
 0.1 RCVD_IN_NJABL          RBL: Received via a relay in dnsbl.njabl.org
                            [24.243.9.220 listed in dnsbl.njabl.org]
Comment 4 Daniel Quinlan 2003-09-23 17:57:35 UTC
What's the IP address(es) of the machine running SpamAssassin?
Comment 5 David L. Crow 2003-09-23 18:09:24 UTC
spamassassin is running on a dual network box that is the firewall for my home
network.  The external interface is orangeblood.org which is currently
66.68.175.25 (another austin.rr.com dynamic/residential host where I host my
domain).  The internal interface is in one of the private address ranges and
is known as waterloo.orangeblood.org on my internal DNS server.
Comment 6 Daniel Quinlan 2003-09-23 19:24:31 UTC
Which exact version of SpamAssassin was run on the message?

I can't reproduce the problem.
Comment 7 David L. Crow 2003-09-23 19:34:43 UTC
2.60.  I built an RPM from the .tar.gz file from
http://www.spamassassin.org/released/Mail-SpamAssassin-2.60.tar.gz using
rpmbuild -ta Mail-SpamAssassin-2.60.tar.gz .

I just downloaded the attachment that I created and piped it through
spamassassin -t and now get an addition "rule hit":

Content analysis details:   (7.6 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.1 HTML_60_70             BODY: Message is 60% to 70% HTML
-1.5 BAYES_01               BODY: Bayesian spam probability is 1 to 10%
                            [score: 0.0130]
 0.1 HTML_MESSAGE           BODY: HTML included in message
 2.6 SUSPICIOUS_RECIPS      Similar addresses in recipient list
 3.5 RCVD_IN_NJABL_DIALUP   RBL: NJABL: dialup sender did non-local SMTP
                            [24.243.9.220 listed in dnsbl.njabl.org]
 2.6 RCVD_IN_DYNABLOCK      RBL: Sent directly from dynamic IP address
                            [Dynamic/Residential IP range listed by]
            [easynet.nl DynaBlock - <http://dynablock.easynet.nl/errors.html>]
 0.1 RCVD_IN_SORBS          RBL: SORBS: sender is listed in SORBS
                            [24.243.9.220 listed in dnsbl.sorbs.net]
 0.1 RCVD_IN_NJABL          RBL: Received via a relay in dnsbl.njabl.org
                            [24.243.9.220 listed in dnsbl.njabl.org]
Comment 8 David L. Crow 2003-09-23 19:39:46 UTC
spamassasin -V doesn't show more than 2.60 or I would have included it.  I see
in an email message header that was processed by SpamAssassin the version as

X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on 
	waterloo.OrangeBlood.org
Comment 9 Daniel Quinlan 2003-09-24 20:16:09 UTC
Can you run spamasassin or mass-check on that message with debugging
turned on and attach the debugging to this message?

I suspect it's something to do with the Received: header code, but I don't
know what it is.
Comment 10 David L. Crow 2003-09-24 20:22:22 UTC
Created attachment 1424 [details]
stderr from "spamassassin -D" on previously attached message.

Attached per your request.
Comment 11 Justin Mason 2003-09-24 21:10:57 UTC
Looks like your relay -- waterloo.orangeblood.org -- has a different IP
depending on who's doing the name resolution.

From where SA is running, it has only the IP 192.168.4.1 -- which is private and
therefore cannot be the external relay (as far as SA can see), but presumably
for ms-smtp-02.texas.rr.com to talk SMTP to it, it must have another public one ;)

I'm presuming there isn't another relay in the middle that's not adding Received
lines, because that would be just odd.

debug: received-header: 'by' waterloo.OrangeBlood.org has reserved IP 192.168.4.1
debug: received-header: 'by' waterloo.OrangeBlood.org has no public IPs
debug: received-header: relay 24.93.36.230 trusted? yes
debug: received-header: 'by' ms-smtp-02.texas.rr.com has public IP 24.93.36.230
debug: received-header: relay 24.243.9.220 trusted? no

The fix is to specify "trusted_networks" so the inference code won't try to
guess it; or to give waterloo a public IP on the internal DNS. trusted_networks
is probably easier.
Comment 12 David L. Crow 2003-09-24 21:30:17 UTC
Your presumption regarding my system configuration is correct and is described
in comment #5.

Adding

  trusted_networks 192.168.4.

to ~/.spamassassin/user_prefs does indeed remove the RCVD_IN_DYNABLOCK rule
trigger.

I'm not sure I understand from the documentation in Conf.pm and the debug output

  debug: received-header: 'by' waterloo.OrangeBlood.org has reserved IP 192.168.4.1
  debug: received-header: 'by' waterloo.OrangeBlood.org has no public IPs

why this is required, though.

If you truly feel that there is no bug here, then I'm happy to add the
trusted_networks config items to my system config files and let this bug
move to the resolved state.
Comment 13 Justin Mason 2003-09-24 23:17:33 UTC
Subject: Re: [SAdev]  Dynamic/Residential IP Range checking is too aggressive 


bugzilla-daemon@bugzilla.spamassassin.org writes:
>I'm not sure I understand from the documentation in Conf.pm and the debug output
>
>  debug: received-header: 'by' waterloo.OrangeBlood.org has reserved IP 192.168.4.1
>  debug: received-header: 'by' waterloo.OrangeBlood.org has no public IPs
>
>why this is required, though.

The "by" host has no public IP addresses -- this generally means that that
host is on an internal network without a connection to the internet
(except in this kind of split-DNS setup).   Hence, if it's on the same
private network as the host SpamAssassin is running on, it must be
a trusted host.   

This is an incorrect assumption though, so SpamAssassin misjudges the
number of untrusted hosts off-by-one, and misses the legit relay
ms-smtp-2.  So the first relay it sees that it thinks is external,
is the dialup addr, hence the RCVD_IN_DYNABLOCK hit.

--j.

Comment 14 Daniel Quinlan 2003-10-01 22:09:14 UTC
*** Bug 2537 has been marked as a duplicate of this bug. ***
Comment 15 Dale Blount 2003-10-02 07:31:41 UTC
Duplicate bug 2537 was mine, I'm attaching a sample message as requested.
Comment 16 Dale Blount 2003-10-02 07:32:41 UTC
Created attachment 1447 [details]
DNSBLs test dialup lists one hop too far
Comment 17 Matt Kettler 2003-10-03 11:44:00 UTC
*** Bug 2543 has been marked as a duplicate of this bug. ***
Comment 18 Justin Mason 2003-10-03 13:07:53 UTC
BTW, Dan -- bug 2543 and bug 2537 are not dups of this.

In this case, it's the split-DNS setup that confuses the automatic
trust-inference algorithm.  We can't avoid that -- since the internal host has
no way of querying the external DNS info.  So specifying "trusted_networks" is
entirely the correct action to take and there is no bug.

They are a different issue where there may be a different bug.

So I've reopened bug 2537 for the issues reported in those 2 bugs, and I'll
resolve this one.