|
|
SA Bugzilla – Full Text Bug Listing |
- Home
- | New
- | Browse
- | Search
- | [?]
- | Reports
- | Help
- | New Account
- | Log In
- | Forgot Password
|
|
|
SA Bugzilla – Full Text Bug Listing |
| Summary: | trusted networks isn't enough for people who use forwarding services | ||
|---|---|---|---|
| Product: | Spamassassin | Reporter: | Rob Mueller <apache> |
| Component: | Libraries | Assignee: | SpamAssassin Developer Mailing List <dev> |
| Status: | NEW --- | ||
| Severity: | normal | CC: | apache, dev |
| Priority: | P5 | ||
| Version: | 3.1.8 | ||
| Target Milestone: | Undefined | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Attachments: |
Add trusted_host option
Code to determine if RDNS name is a dialup/dsl |
||
Created attachment 3879 [details]
Add trusted_host option
The code we use that overwrites existing SA methods to add trusted_host
Unfortunately Mail::SpamAssassin::Message::Metadata::parse_received_headers
doesn't really provide a way to easily hook in a new trust system, so I had to
reproduce the whole function with really just a small change, these lines:
# trusted_networks matches?
if ($in_trusted && $did_user_specify_trust && !$self->is_trusted($relay,
$trusted))
{
$in_trusted = 0; # we're in deep water now
}
It would be nice to actually change parse_received_headers so there are more
"hook" places people could get into if they want to change the overall
algorithm in the future as well...
Created attachment 3880 [details]
Code to determine if RDNS name is a dialup/dsl
We use this code on our systems to determine if a machine is a dialup/dsl host
(eg more dodgy and subject to more checks like greylisting, enumeration
detection, etc). Empirically, it seems to be pretty accurate these days
assuming that the machine has any rdns at all. If not, we assume it's on the
dodgy list.
I'm pretty keen on this. aiming at 3.3.0 btw (In reply to comment #0) > The trusted_networks allows you to define the IP addresses of networks you > trust. Consider this situation: > > spam-zombie -> zoneedit-mail-forwarding -> fastmail.fm-customer > > So fastmail.fm runs SpamAssassin. We set our known internal networks as > trusted_networks. This means the relay analysis algorithm will use the > zoneedit-mail-forwarding IP as the -lastexternal IP for some RBL tests such as: > > header RCVD_IN_XBL eval:check_rbl('sblxbl-lastexternal', 'sbl-xbl.spamhaus.org.' > > This isn't very useful, we really want to scan further back to test against the > source IP. Now the way to do this is to add all the known forwarding systems to > your trusted_networks. hmm, hold on. "trusted" != "internal" -- I think you're conflating trusted_networks and internal_networks here. I haven't looked at the code, but this is something I've had in mind for a while. I wanted to pluginize most of Received.pm first though. I could see this ending up in a 3.2.x release sometime in the summer though. The SRS support part needs to end up in the SPF plugin though, it's not something we should screw about with while determining trusted/internal though. (In reply to comment #4) > hmm, hold on. "trusted" != "internal" -- I think you're conflating > trusted_networks and internal_networks here. Maybe, maybe not. In his described case they'll be the same anyway. It really has nothing to do with adding support for using rDNS to extend trust though. BTW... I meant I could see this implemented in Received.pm during 3.2, pluginizing would be 3.3. I don't think it's in time to get into 3.2.0; at this stage we should consider that release in feature-freeze mode, IMO. 3.3.0, I'm fine with. Yeah, not now, but maybe a small patch to 3.2.x later... (In reply to comment #5) > I could see this ending up in a 3.2.x release sometime in the summer though. I was about to say "no", but then I realised that it doesn't affect _existing_ behaviour -- it's a new feature driven entirely by a new config keyword. so maybe ;) Sorry, I'm pulling in this quote from http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5374 because I think it's more related to this and should respond here: > FWIW, I think extending trusted_networks towards the sender further than > internal_networks is pointless 99.99% of the time (your local DNS cache absorbs > any penalty of a few extra DNS lookups in systems busy enough for the penalty to > be an issue). I recommend that people don't do it. The only place I see > differing trusted/internal configs is on the MSA side where you can't determine > auth from the headers (or POPAuth). With msa_networks now implemented it's not > even necessary in this case anymore. In hindsight, I think I'm agreeing with this. I think I was confused about the use of -lastexternal, and now I'm not convinced that my patch here actually adds anything useful to DNSBL checks. So I went back and checked some more to see what it was helping on, and really it seemed to be 2 cases: 1. Forwarder hosts that for some reason were on an RBL. By extended trust into them, their IPs aren't checked, and thus when some persons forwarding service ends up being RBL listed for some reason, suddenly a lot of their email was being marked as spam 2. SPF checks for forwarding services that don't use SRS because I have some other code that overrides Mail::SpamAssassin::Plugin::SPF::_get_relay to check each relay for ->{trust_type}. Apart from that though, I'm not sure any of this is actually a help, and I may have been leading things up the garden path. Still, things that reduce false positives and overall accuracy for users is always a good thing. Rob moving most remaining 3.3.0 bugs to 3.3.1 milestone reassigning, too moving all open 3.3.1 bugs to 3.3.2 Moving back off of Security, which got changed by accident during the mass Target Milestone move. I'd also like to see *_networks work with rdns names. Obviously this isn't happening with 3.3.. target undefined until someone wants to tackle it. |
The trusted_networks allows you to define the IP addresses of networks you trust. Consider this situation: spam-zombie -> zoneedit-mail-forwarding -> fastmail.fm-customer So fastmail.fm runs SpamAssassin. We set our known internal networks as trusted_networks. This means the relay analysis algorithm will use the zoneedit-mail-forwarding IP as the -lastexternal IP for some RBL tests such as: header RCVD_IN_XBL eval:check_rbl('sblxbl-lastexternal', 'sbl-xbl.spamhaus.org.' This isn't very useful, we really want to scan further back to test against the source IP. Now the way to do this is to add all the known forwarding systems to your trusted_networks. But to do that, you'd have to know the internal IP arrangement of every place like gmail, yahoo, hotmail, zoneedit, pobox, etc, not really feasible. Instead a better idea is to add some trusted_hosts to the list. In this case, we will "trust" a set of hosts if: 1. We think all their servers are secure and zombie free 2. We know they don't have any dialup/dsl customers with RDNS that contains their hostname So we have a new configuration option that looks like: trusted_host pobox.com If when scanning back through a Received header, we see an RDNS that ends with pobox.com, we then do a forward lookup on the name, and if it matches the IP address in the Received header, we assume the header isn't forged, and because we trust pobox.com servers, we set that Received header as trusted. This means we can add trust for whole providers with one configuration option without having to know the internal IPs of that provider. For the moment, I haven't created a patch for this, instead I've implemented it as hooks that replace and override existing code (see attached). Additional options 1. The trusted_hosts configuration option has an optional second parameter. trusted_host pobox.com srs When srs is added, it means we know the host supports SRS forwarding, we use that in the Mail::SpamAssassin::Plugin::SPF to change how _get_relay works so it doesn't scan back to the first trusted host based on the new above logic. Also in attached code. 2. With this system, you can NOT had ISPs to your trusted_hosts set because most ISPs have correct forward/reverse DNS for their dialup/dsl hosts, which are the main things we're trying to check against. A solution to this would be to try and detect dialup/dsl hosts based on their rdns hostname. This is actually possible to do fairly accurately with a bit of regexp hackery (see attached) If this code were used as an exception to the is_trusted_relay test, then you could also add ISPs to the trusted_hosts list. 3. Rather than each version/user of SA having to maintain a trusted_hosts list, a better idea would be to have a URIWL that you could query against that would maintain known good hostnames. Then that could be maintained separately to each version of spamassassin. No code for that yet...