Created attachment 1372 [details]
Message headers and SA report that triggered this bug. Personal info removed.

These is the headers and SA report of an SMTP Auth message that triggered this
rule. Unfortunately it was forwarded from Eudora and is somewhat rewrapped.

This bug also creates a problem for ANYONE on dynamic IP space. As an example,
if I'm on a dynamic IP and relay through my ISP's mail server, my ISP will add a
Received: line with my dynamic IP in it. SA is picking up on that even though
it's the first hop! SA should only look at the topmost Received line by default
and not any Received lines further down (which can easily be forged anyway...)
The false positives with RCVD_IN_DYNABLOCK (and friends) are huge at this point.

Is the machine running SA using a private  (ie: 192.168.*.*) IP address?

If so, this is a dupe of bug 2537, and there's a workaround in that bug (manualy
setting trusted_relays).

I have encountered this bug also. A client using SMTP AUTH over TLS sent an
email and triggered 4 rules - sending it over the top of my 6 allowed points.
They are on an ADSL line, and authenticate over the net with a real and
routeable IP as allocated by their ISP. I have removed any personal info from
this fragment - I can provide the full headers for you if needed.

Received: from *******.co.uk (*-*-*-*.dyn.*.co.uk [*.*.*.*])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by *.*.net (Postfix) with ESMTP id B40B45C1B1
	for <*@*****.net>; Mon,  8 Mar 2004 10:51:26 +0000 (GMT)
X-Spam-Status: Yes, hits=7.0 tagged_above=0.0 required=6.0
 RCVD_IN_DYNABLOCK, RCVD_IN_NJABL, RCVD_IN_NJABL_DIALUP, RCVD_IN_SORBS

To address Matt Kettler's comment (sorry I didn't see it before now), no the
machine running SA is NOT on private IP space.

As a work-around, it is possible to craft a custom rule to look for your own
SMTP-AUTH strings in the header, and add a negative value in that case.  This
rule has to be customized for each site. IE, for my site, based on the headers I
originally posted, this is what I use:

header L_SMTP_AUTH      Received =~ /^from [^ ]+ \([^)]*\)\s+\(authenticated
bits=\d+\)\s+by westnet/
describe L_SMTP_AUTH    Message was received locally via an SMTP AUTH connection.
score L_SMTP_AUTH       -20
                                                                               
                                       

this is a problem.  we have no way of telling that the message was submitted via
SMTP AUTH.  AFAIK, there's no standard format for that data; see the "TLSv1" and
"authenticated-bits" snippets in those log lines.  How can we tell if the user
really did auth?

We can't blindly trust "TLSv1" tokens, since once spammers add the openssl libs
to their ratware, they can connect to MXes using TLS and use unauthenticated,
but encrypted, spam connections.  

(however the "authenticated-bits" token may be useful to guess this, although
I'm mystified why that's "0".)

One way around this problem is to *not* scan mail submitted by authed users;
just scan the stuff coming in from non-authenticated SMTP connections from the
outside world.  (having a separate "outgoing" MTA and an "incoming" MX MTA would
do this.)

*** Bug 2537 has been marked as a duplicate of this bug. ***

I posted the SMTH AUTH headers for sendmail.  How hard would it be to have rules
for at least the major MTAs (sendmail, postfix, qmail, etc).

In configuration, a site useing SMTP AUTH would select their MTA. For sendmail
at least, the hostname of the AUTH machine is also necessary. Haveing that in
the rule makes each site's rule unique, and difficult to forge.

This would be a general form of the 'hack' I posted.

Is this a reasonable approach ?

This still won't fix the pop-before-smtp problem as it leaves no extra headers
and is still in quite widespread use.  Can someone remind me why SA doesn't just
always skip scanning the first IP in Received when the notfirsthop flag is set?

Something like, skip scanning first hop unless it is the only other hop in the
message?

'Can someone remind me why SA doesn't just always skip scanning the first IP in
Received when the notfirsthop flag is set? Something like, skip scanning first
hop unless it is the only other hop in the message?'

That is *exactly* what it does -- and it will still produce FPs in this
situation.   To recap, here's what this bug is describing

1. user authenticates with your MTA, from a dynamic IP address:
dynamic-pool.someisp.net

2. user submits a msg to an address @ your MTA: your-mta.you.net

3. your MTA scans the message, since the message is for delivery to a local user

4. the only Received hdr is one saying

   Received: from dynamic-pool.someisp.net ....	by your-mta.you.net

There are no other Received hdrs.  By the rules of notfirsthop, we *HAVE* to use
that IP for the Dynablock lookup, as you yourself describe.

5. it hits.

(I think comment #2 is are not talking about this case.  It sounds like some
REceived hdr format is not being parsed by SA -- which causes dynablock FPs.  
But that is a different bug, if it is a bug -- no samples or header data was
posted, so I have no idea.  *This* bug is about the SMTP AUTH thing!)


The correct response is to:

1. for the POP-before-SMTP case, allow SA to give bonus "nice" points to IPs
that have authed via POP, bug 3086.  (If you're talking about POP-before-SMTP,
best to go there.)

2. match header data that indicates that the submitter used SMTP AUTH.  That's
this bug.   And I agree with Chris Candreva, we need more samples from the other
MTAs, and this may be a reasonable approach.

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg submission

> 1. for the POP-before-SMTP case, allow SA to give bonus "nice" points
> to IPs that have authed via POP, bug 3086.  (If you're talking about
> POP-before-SMTP, best to go there.)

Shouldn't it still be up to the MTA to tag the Received: line as
authorized?  I suppose POP-before-SMTP could be done with some front-end
making the information unavailable to the MTA. 
 
> 2. match header data that indicates that the submitter used SMTP AUTH.
> That's this bug.  And I agree with Chris Candreva, we need more
> samples from the other MTAs, and this may be a reasonable approach.

If by (2), you mean we should just handle authorized hosts in one of the
following ways: trust them, don't include them in notfirsthop testing,
or consider them non-dynamic, then I agree.

Daniel

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg submission 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>> 1. for the POP-before-SMTP case, allow SA to give bonus "nice" points
>> to IPs that have authed via POP, bug 3086.  (If you're talking about
>> POP-before-SMTP, best to go there.)
>
>Shouldn't it still be up to the MTA to tag the Received: line as
>authorized?  I suppose POP-before-SMTP could be done with some front-end
>making the information unavailable to the MTA. 

That's how it is done currently -- there's several scripts that tail
the POP/IMAP logfile and update an access DB file.

>> 2. match header data that indicates that the submitter used SMTP AUTH.
>> That's this bug.  And I agree with Chris Candreva, we need more
>> samples from the other MTAs, and this may be a reasonable approach.
>
>If by (2), you mean we should just handle authorized hosts in one of the
>following ways: trust them, don't include them in notfirsthop testing,
>or consider them non-dynamic, then I agree.

meaning don't include them in nfh testing.

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFAaJe7QTcbUG5Y7woRApnqAJ9ntutP9pMidkukeenV7dJdr9dD+QCgzuzs
PzWjhp0cBbDq0j/NYE0xaN8=
=JHdV
-----END PGP SIGNATURE-----

Why can't we have a configuration option that lists the local MTAs that only
accept mail from trusted clients? Then there would be no need to parse various
SMTP_AUTH indicators or do anything special for POP-before-SMTP, and it would
handle any other authorization method. The effect of the option would be to
disable the one Received header rule of notfirsthop when the MTA is in the list.

The simplest form of the option would be to accept all, i.e, disable the one
Received header header to notfirsthop. That would be used when the delivering
MTA does not accept unauthorized connections from outside the network.

This will not help if an ISP uses the same MTA to receive mail from other MTAs,
to receive mail from authorized clients, and to deliver the mail locally. But at
that point I would say it is their configuration problem. If they want a way to
distinguish between mail that arrives authenticated and mail that arrives
unauthenticated they should logically separate the server that they tell users
to use to send mail and the ones that their MX records point to.

Re comment 13:

You suggestions (especially the last paragraph) imply that everybody must have 
two mailservers one for users and one for MX records. Do you really want to 
force small companies with their own hosting and with only 10 users to buy a 
second mailserver for this?

 

> force small companies [ ... ] to buy a second mailserver

I said they should _logically_ separate the servers. That can be done on one
computer with two ip address aliases on the same network interface. Or use one
ip address with an authentication method that uses a different port such as
using SMTP/TLS or a VPN instead of STARTTLS or POP-before-SMTP. This isn't just
a SpamAssassin issue. You use one ip address as your domain's public MX server,
and a different ip address as the authenticating mail relay for your
users/employees. That makes it trivially easy to separate handling of possible
outide spam from mail that your users send. If you use one ip address for both,
you have a difficult problem. If your setup is so small and low budget that you
cannot have two static ip addresses and you have to use STARTTLS or
POP-before-SMTP, you probably should be using your ISP's mail server as the MX
server for your domain, even if you want to do the bulk of the mail processing
on your own server. That indirection will also take care of this problem.

yeah - it's the addition of SMTP AUTH/POP-before-SMTP, without separation of
incoming and outgoing servers, that makes this hard.

"Doctor, Doctor, it hurts when I do this"

"Then don't do that"

If using one server for three different functions makes it so difficult, I would
suggest that they separate the functions into separate (virtual) servers rather
than that SA get imperfectly kludged to look up temporary POP access ip
addresses and parse various SMTP/AUTH indicators.

Well, maybe I'm giving up too easily and somebody will come up with an elegant
solution to the problem.

"yeah - it's the addition of SMTP AUTH/POP-before-SMTP, without separation of
incoming and outgoing servers, that makes this hard."

Justin, if the servers *are* separate phyiscal servers and one of the above
methods is in use, how is it supposed to be configured? Should trusted_networks
not list the outgoing MTA?  If it is listed in trusted_networks, it seems to
skip that hop too and still test the dynamic client.

'Justin, if the servers *are* separate phyiscal servers and one of the above
methods is in use, how is it supposed to be configured? Should trusted_networks
not list the outgoing MTA?  If it is listed in trusted_networks, it seems to
skip that hop too and still test the dynamic client.'

With separate servers for incoming (MX) and outgoing (MSA), when an
authenticated user connects to the MSA, their mail is delivered to user accounts
without spam scanning being applied.   Only messages arriving from unauth'd
internet clients are scanned.

more accuracy and performance bugs going to 3.1.0 milestone

Correct me if I'm wrong, but 'changing' the _first_ untrusted relay into a
trusted relay when "(authenticated" is found between the 'from' and 'by' info
should take care of the problem.

We can do this since we (hopefully) trust our own server to only add
'authenticated' when the relay (whether it's an MUA or even another MTA) is in
fact authenticated.

Sendmail and MDaemon adds this authenticated line by default.  I believe Exim
needs to be configured to add the authenticated line.


Example of an auth'd email directly from an MUA to a system's one and only mail
server:

Received: from mycomputer (dynamic-ip.isp.net [1.2.3.4])
        (authenticated user/bits/whatever)
        by ourdomain.com (smtp.ourdomain.com [5.6.7.8])
        with whatever
        for <whoever>; Some Date
From: authenticateduser@wherever.com
To: localuser@ourdomain.com
Date: Whenever
Subject: blah!


You can't forge the above (since you'd only look at the first untrusted relay
received header, which is going to be added by your own server), so long as your
server isn't wide open and authenticating anyone who asks.


From Justin Mason  2004-03-29 11:26
> There are no other Received hdrs.  By the rules of notfirsthop, we *HAVE* to 
> use that IP for the Dynablock lookup, as you yourself describe.

Of course, since we trust this host authenticated relay we *DO NOT HAVE TO* use
that IP for Dynablock lookup.  We only have to do dynablock lookups when there
is a single received header and that user isn't auth'd since it could be a spam
host zombie/etc connecting directly to our MTA sending mail for us.

In fact, if there's only the single received header (which will be the case most
of the time, and maybe we should only convert the auth'd relay to a trusted
relay if there would normally only be a single relay -- there are cases when
there are multiple relays with the first untrusted one auth'd), ALL_TRUSTED
should fire and we could really short circuit, someday.

Created attachment 2540 [details]
trust smtp auth implementation

Rather trivial.

This prevents Dynablock lookups/etc of (non-forged) authenticated relays by
treating it as trusted/internal.  It'll also (correctly) cause ALL_TRUSTED to
fire for roaming, but authenticated, users.

The POP before SMTP issue still exits, although POP before SMTP has got to be
the most annoying method to send mail anyway (since most clients try to send
mail before they do a POP check), so those people affected will probably commit
suicide before too much of their mail is flagged as spam.

Our servers use RFC 3848 protocol names in the "with" part of Received: lines,
so authenticated SMTP is either "with esmtpa" or "with esmtpsa". For example:

Received: from cyan.csi.cam.ac.uk ([131.111.10.17]:4561)                        
    by ppsw-3.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.153]:25)          
    with esmtpsa (PLAIN:fanf9) (TLSv1:DHE-RSA-AES256-SHA:256)                   
    id 1CYjFm-0003c4-AB (Exim 4.44) for fanf2@cam.ac.uk                         
    (return-path <fanf9@hermes.cam.ac.uk>); Mon, 29 Nov 2004 10:59:51 +0000     

unauthenticated Received: lines look like

Received: from mailgate-internal1.sri.com ([128.18.84.103]:58814)               
    by ppsw-2.csi.cam.ac.uk (mx.cam.ac.uk [131.111.8.142]:25)                   
    with smtp id 1CY9kb-00069x-7N (Exim 4.44) for dot@dotat.at                  
    (return-path <risks-bounces+dot=dotat.at@csl.sri.com>); Sat, 27 Nov 2004
21:05:01 +0000                                                  

I have just applied this patch on my 3.0.1 installation. So far it is working as
intended.

Created attachment 2541 [details]
trust smtp auth implementation -- compliant with RFC 3848

This patch is RFC 3848 aware, which will cover any MTA that includes any of
ESMTPA, ESMTPSA, LMTPA, or LMTPSA (case insensitive) in the with line.

ESMTPS and LMTPS is not accepted since that's only for TLS.

I submitted a CLA today.  Any chance this can be squeezed into 3.0.2?  The
patch is really simple.  Being able to trust auth'd users allows for a nice
increase in the various dynablock scores.

*** Bug 3404 has been marked as a duplicate of this bug. ***

Daryl, you're exactly right.

If the last untrusted header contains header strings indicating that the user
authenticated to the trusted server, we can indeed extend the trust boundary to
include the next hop.

So the next question is -- can we differentiate (a) spammers connecting to a
server with TLS, then relaying unauthenticated spam, from (b) a real user
connecting to a server with TLS and authenticating?   I know the idea of
spammers taking the performance hit to do a TLS handshake *seems* unlikely, but
given the CPU power of the zombie networks nowadays, it's really not that
unlikely in future, so I think we will see spam that does (a).

(there's already spam that *forges* Received lines claiming to do (a), but
that's not an issue as that's >1 line into the untrusted header set.)

a TLS connection without user authentication looks like this, afaics:

Received: from ... (... [209.204.169.179]) (authenticated bits=0) by
smtp1.supranet.net (8.12.10/8.12.10) with ESMTP id iAUCaOSD095285
(version=TLSv1/SSLv3 cipher=DES-CBC3-SHA bits=168 verify=NOT); Tue, 30 Nov 2004
06:36:25 -0600 (CST)

or

Received: from ... (... [172.24.77.106])
        (authenticated bits=0)
        by ... (8.12.11/8.12.11) with ESMTP id i98HCirF017491
        (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO)
        for <....>; Fri, 8 Oct 2004 10:12:44 -0700

and one with user auth looks like this:

Received: from ... ([...])
        (authenticated user dos@example.com)
        by smtp.film-tech.net (smtp.film-tech.net [66.98.221.156])
        (Cipher TLSv1:RC4-MD5:128) (MDaemon.PRO.v6.8.5.R)
        with ESMTP id 44-md50000000012.tmp
        for <...>; Tue, 09 Nov 2004 20:35:57 -0600

AFAICS, the current patch doesn't differentiate them.

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

Justin, this is what a TLS only connection looks like on Sendmail 8.12.8:

Received: from [192.168.123.141] (hammer.dostech.net [192.168.123.141])
	by cyan.dostech.net (8.12.8/8.12.8) with ESMTP id iB27qMRC009486
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <...>; Thu, 2 Dec 2004 02:52:34 -0500


I think the first two examples you posted above are of auth'd TLS 
connections.  The last one is of an Windows NT MDaemon server I have the 
(surprisingly easy) displeasure of running.


Daryl

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

Here's an authenticated TLS connection to the same server:

Received: from [192.168.123.141] (hammer.dostech.net [192.168.123.141])
	(authenticated bits=0)
	by cyan.dostech.net (8.12.8/8.12.8) with ESMTP id iB29oceg011436
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <...>; Thu, 2 Dec 2004 04:50:46 -0500


So, to answer your question Justin, I think we can indeed tell them 
apart by the presence of 'authenticated bits' (in the case of Sendmail) 
only appearing in connections which are authenticated, regardless of 
TLS.  The existing patch determines this correctly.

Additionally, I think 'HTTP' should be added to the list (ESMTPS, 
ESMTPSA, LMTPS, LMTPSA) of authenticated 'with' methods in the patch 
(see bug 4008) since I can't think of any reason why a non-authenticated 
session would include 'with HTTP' in the first untrusted relay header, 
save for a bad server configuration.


Daryl

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

On Thu, 2 Dec 2004 bugzilla-daemon@bugzilla.spamassassin.org wrote:
>
> Additionally, I think 'HTTP' should be added to the list (ESMTPS,
> ESMTPSA, LMTPS, LMTPSA) of authenticated 'with' methods in the patch
> (see bug 4008) since I can't think of any reason why a non-authenticated
> session would include 'with HTTP' in the first untrusted relay header,
> save for a bad server configuration.

I think ESMTPS and LMTPS should be removed, since they are (probably) not
authenticated. (This is assuming that client cert authentication isn't
being used, but most MUAs have really shitty support if any for client
certs so I expect it won't be a problem.)

Tony.

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

You're right, I meant to write ESMTPA and LMTPA.  ESMTPS and LMTPS 
aren't on the list since they are, as I noted the other day, only TLS 
connections.

Created attachment 2544 [details]
trust smtp auth implementation -- compliant with RFC 3848 and more

Adds 'with ASMTP' (Authenticated SMTP) used by Earthlink and also spotted in
some Exim 4.34 headers.

Also adds 'with HTTP' for webmail servers.

Any more comments on this patch?


Daryl

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg submission 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I still need to convince myself that this won't (incorrectly) give trust
to simple, unauthenticated server-to-server TLS-encrypted hops.
I don't really know enough about how the various MTAs log that
data to be sure, yet... more sample headers would probably help
convince me ;)

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFBr6SrMJF5cimLx9ARAk65AJwJvPFIww3QaeDi5PbxXvcWqQZbUQCgikWW
otBFlYiykZQK4vm9EIbysmE=
=zHHJ
-----END PGP SIGNATURE-----

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

Well, any related headers either of us find will be interpreted based on 
what we both already believe about non-auth+TLS/auth+TLS headers.  So 
that leaves us with having people provide non-auth+TLS and auth+TLS 
headers that they personally know to be true (such as the ones I posted 
above).

Should a request be made for such on the users@ list?

Do we all agree on the validity of the authenticated nature of the 
'with' types in the patch (which are based in part on RFC 3848)?


Daryl

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

An authenticated example from the SpamAssassin Wiki, which I suspect was 
added by Dawn Keenan of Information Systems Technology at the University 
of Waterloo, is at http://wiki.apache.org/spamassassin/DynablockIssues

Received: from 192.168.2.125 
(CPE0004e24b9419-CM000a7365d82c.cpe.net.cable.rogers.com
       [63.139.187.25]) (authenticated (0 bits)) by 
services04.student.cs.uwaterloo.ca
       (8.11.7/8.11.7) with ESMTP id hA41X1B23955
       for <recipient@example.org>; Mon, 3 Nov 2003 20:33:03 -0500 (EST)


Dawn's last email to the users list also appears to have a TLS-only hop 
(note the lack of 'authenticated bits'):

Received: from ist.uwaterloo.ca (localhost [127.0.0.1])
	by ist.uwaterloo.ca (8.12.11/8.12.11) with ESMTP id iAOEj42R025681
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Wed, 24 Nov 2004 09:45:05 -0500 (EST)
Received: (from dkeenan@localhost)
	by ist.uwaterloo.ca (8.12.11/8.12.7/Submit) id iAOEj1Qe025550;
	Wed, 24 Nov 2004 09:45:01 -0500 (EST)


Also note the "Received: Header" section of 
http://www.sendmail.org/~ca/email/starttls.html only notes the addition of

(using ${tls_version} with cipher ${cipher} (${cipher_bits} bits) 
verified ${verify})

or

(version=${tls_version} cipher=${cipher} bit=${cipher_bits} 
verify=${verify})

in TLS sessions.  There's no mention of an 'authenticated bits' token. 
The document does specify that a user can authenticate with TLS, and 
that this would be indicated by 'verify=OK' or 'verified OK' appearing 
in the TLS line, instead of 'verify=NO' or 'verified NO'.

Looking at my stock sendmail.cf file and various Sendmail version README 
files, I don't see anything to suggest that a TLS session would ever add 
an 'authenticated bits' token.


So, I think it's fair to say that 'authenticated bits' won't appear in 
non-authenticated TLS sessions and that a regex should be added to the 
patch to catch the TLS verified authentications noted above.


Daryl

Subject: Re:  detect SMTP AUTH to avoid dynablock FPs on legit msg
 submission

> I still need to convince myself that this won't (incorrectly) give trust
> to simple, unauthenticated server-to-server TLS-encrypted hops.

WRT Exim, the old "asmtp" and the new "esmtpa"/"esmtpsa"/"lmtpa"/"lmtpsa"
tokens are only used for messages received after successful SMTP AUTH.

Tony.

ok, indeed, the "(authenticated bits=" pattern only appears when a user uses
auth+TLS -- I checked my corpus in a bit more detail.  Also, the other patterns
(RFC3848, esmtpa/lmtpa, and HTTP) seem safe too.

I think this is very nearly good to go... +0.5.   However I would suggest one
modification -- the $auth value shouldn't contain spaces, which it will in the
first (sendmail/MDaemon) case.  Since it's only used as a boolean, I'd suggest
either:

  - dealing with the sendmail case by using the static string "sendmail", or
similar, instead of the caught text
  - changing it to just be the integer 1 instead of the string caught.

I think the former's preferable.

oh -- regarding the sendmail data you dug up -- I think "verification" there
refers to client TLS certificate verification, which AFAIK will be quite rare. I
wouldn't stall the patch waiting for that addition if the "authenticated bits"
pattern catches the auth+TLS+verified-client-cert case, anyway.


(PS: for future reference -- here's a few more samples of authenticated
handovers from my greps through the ham mail corpus:

  from 153.90.199.141        (SquirrelMail authenticated user admin);
    by web1.cs.montana.edu with HTTP;        Thu, 23 Sep 2004 14:35:29 -0600 (MDT)
  from [192.168.1.3] (80-28-223-208.adsl.nuria.telefonica-data.net [80.28.2
23.208]) (authenticated bits=0) by mac.com (Xserve/smtpin08/MantshX 4.0) with
ESMTP id i8NIdH8G002812 for ...
  from perceptions.couk.com (81.103.146.112) by n082.sc1.cp.net (7.0.030.2)
(authenticated as r.dickenson) id 414B418B002D65F1 for forteana [MUNGED]
yahoogroups.com; Thu, 23 Sep 2004 18:42:17 +0000
  from 141.44.167.13 (p83.129.191.197.tisdip.tiscali.de [83.129.191.197])
(authenticated bits=0) by sunny.urz.uni-magdeburg.de (8.12.10/8.12.10) with
ESMTP id i8ND9v0N017746 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO)
for <ilug [MUNGED] linux.ie>; Thu, 23 Sep 2004 15:09:59 +0200
  from [10.41.4.7] (10.41.4.7) by puma.dub0.ie.cp.net (7.2.033.1) (authenti
cated as gary.coady) id 414ED0B400006D01 for boxhosting [MUNGED] diva.ie; Fri,
24 Sep 2004 13:44:40 +0100
  from rousalka.dyndns.org (81.64.155.54) by mx.laposte.net (7.0.028)
(authenticated as Nicolas.Mailhot) id 413489B100C9C1FD for fedora-devel-list
[MUNGED] redhat.com; Tue, 28 Sep 2004 21:43:43 +0200
  from [10.0.0.253] (82-68-189-22.dsl.in-addr.zen.co.uk [82.68.189.22])
(authenticated (0 bits)) by ensim.rackshack.net (8.11.6/8.11.6) with ESMTP id
i8TAFAI25021 for <discuss [MUNGED] lists.surbl.org>; Wed, 29 Sep 2004 10:15:10 GMT 
  from [213.174.165.187] (213.174.165.187) by vsmtp1.tin.it (7.0.027) (auth
enticated as mgiammarco [MUNGED] virgilio.it) id 416A525B0000A53B for
linux-thinkpad [MUNGED] linux-thinkpad.org; Mon, 11 Oct 2004 12:52:46 +0200
  from [10.10.10.215] (Collation_Software.demarc.cogentco.com [66.250.6.18])
(authenticated bits=0) by waste.org (8.12.3/8.12.3/Debian-6.6) with ESMTP id
i46MehGO005108 for <fork [MUNGED] xent.com>; Thu, 6 May 2004 17:40:44-0500
  from ausisaps301-dmz.aus.amer.dell.com ([143.166.226.16]) (SquirrelMail
authenticated user hoolis); by www.penguintowne.org with HTTP; Mon, 22 Mar 2004
12:54:13 -0600 (CST)
  from dsl-082-082-143-115.arcor-ip.net (dsl-082-083-139-045.arcor-ip.net
[82.83.139.45]) (authenticated bits=0) by postman.arcor.de
(8.13.0.PreAlpha4/8.13.0.PreAlpha4) with ESMTP id i2U75jD1003350 for
<linux-thinkpad [MUNGED] linux-thinkpad.org>; Tue, 30 Mar 2004 09:05:45 +0200 (MEST)
  from [192.168.111.8] (gw.local.linuxlobbyist.org [192.168.111.1])
(authenticated bits=0) by iadonisi.to (8.12.11/SQL-8.12.11-5/8.12.11) with ESMTP
id i4S6SML6017129 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256
verify=NO) for <spf-discuss [MUNGED] v2.listbox.com>; Fri, 28 May 2004 02:28:23
-0400
  from p5483b7c0.dip.t-dialin.net ([84.131.183.192] helo=192.168.1.23) by
moonflower.de with asmtp (TLS-1.0:RSA_ARCFOUR_MD5:16) (Exim 4.34) id
1CIoQP-0006SN-GV for linux-thinkpad [MUNGED] linux-thinkpad.org; Sat, 16 Oct
2004 15:16:47 +0200
  from bgp01132961bgs.ypeast01.mi.comcast.net ([68.42.119.201]
helo=moonweaver.home.awesomeplay.com) by outbound.mailhop.org with esmtpsa
(TLSv1:RC4-SHA:128) (Exim 4.42) id 1CJic5-00067m-U7


I couldn't find a single nonauth+TLS handover that used "(authenticated".   For
reference, here's headers that used nonauth+TLS:

  from dsl092-076-133.bos1.dsl.speakeasy.net ([66.92.76.133]
helo=pendaran.arborius.net) by sc8-sf-mx1.sourceforge.net with esmtp
(TLSv1:AES256-SHA:256) (Exim 4.41) id 1CIlfc-0003Pa-8W for xvoice-user [MUNGED]
lists.sourceforge.net; Sat, 16 Oct 2004 03:20:18 -0700
  from gilmore.ael.be ([158.64.60.71]) by castlerea.stdlib.net with esmtp
(TLSv1:DES-CBC3-SHA:168) (Exim 4.41) id 1CIusZ-00049K-45 for e-voting [MUNGED]
lists.stdlib.net; Sat, 16 Oct 2004 21:10:16 +0100 
  from rc3.isc.org (rc3.isc.org [IPv6:2001:4f8:3:bb::25])       (using TLSv1
with cipher DHE-RSA-AES256-SHA (256/256 bits))        (No client certificate
requested)  by sf1.isc.org (Postfix) with ESMTP id C986F284EE       for <jm
[MUNGED] jmason.org>; Sat, 16 Oct 2004 21:30:02 +0000 (UTC) (envelope-from
bind-users-bounce [MUNGED] isc.org)
  from rubel.csumb.edu (rubel.csumb.edu [198.189.237.214]) (using TLSv1 with
cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by
sf1.isc.org (Postfix) with ESMTP id 23587284EE for <bind-users [MUNGED]
isc.org>; Sat, 16 Oct 2004 23:32:19 +0000 (UTC) (envelope-from snort [MUNGED]
csumb.edu)
  from p50894de7.dip0.t-ipconnect.de ([80.137.77.231]:11218 helo=sandpiper) by
mail1.isc.de with esmtp (TLSv1:RC4-SHA:128) (Exim 4.04) id 1CJaZW-0006rU-00 for
linux-thinkpad [MUNGED] linux-thinkpad.org; Mon, 18 Oct 2004 18:41:23 +0200
   

note: @ replaced with " [MUNGED] " throughout.)

Created attachment 2547 [details]
trust smtp auth implementation -- compliant with RFC 3848 and more

$auth set to 'Sendmail' in first case.

+1 from me for 3.0.2 and 3.1.0.  moving to 3.0.2 queue for further votes

+0.9 on 2547 with the following questions without answers:  
  * There are some people who run SpamAssassin at SMTP-time.  Did the server 
add his own Received header then? 
  * I've been told that ISS in some versions ships an SMTP server and some 
HTTP relays which are enabled per default (yikes!).  Could 'HTTP' misfire for 
such open relays? 

Subject: Re:  [review] detect SMTP AUTH to avoid dynablock FPs on
 legit msg submission

On Thu, 9 Dec 2004 bugzilla-daemon@bugzilla.spamassassin.org wrote:
>   * There are some people who run SpamAssassin at SMTP-time.  Did the server
> add his own Received header then?

In the case of Exim, yes.

>   * I've been told that ISS in some versions ships an SMTP server and some
> HTTP relays which are enabled per default (yikes!).  Could 'HTTP' misfire for
> such open relays?

No, the Received: with HTTP is added by the webmail user agent, not your
common-or-garden HTTP proxy.

Tony.

*** Bug 3496 has been marked as a duplicate of this bug. ***

Subject: Re:  [review] detect SMTP AUTH to avoid dynablock FPs on
 legit msg submission

> ------- Additional Comments From spamassassin-contrib@msquadrat.de
> +0.9 on 2547 with the following questions without answers:  
>   * There are some people who run SpamAssassin at SMTP-time.  Did the server 
> add his own Received header then? 

If it doesn't it should, this isn't the only case where SpamAssassin 
relies on a milter to at least spoof a received header.  See bug 2860.


>   * I've been told that ISS in some versions ships an SMTP server and some 
> HTTP relays which are enabled per default (yikes!).  Could 'HTTP' misfire for 
> such open relays?

It's not SpamAssassin's job to protect against open relays.  I believe 
the ISS HTTP relays are authenticated though, anyway.  If they're not 
they should be shut off, or made that only trusted hosts can connect to 
them.


Daryl

reminder: this needs votes

+1

+1

Taking, to apply.

Hmmm...this patch doesn't pass tests on the 3.0 branch.  So, I'm removing my +1.
 Does someone have a cleaner patch against 3.0?

I think the received-parser test suite was not updated.  I'll take a look.
FWIW, if it's just the addition of "auth=" to most of the patterns, and it
doesn't throw up a bug, I'll just use the existing +1's ;)

Subject: Re:  [review] detect SMTP AUTH to avoid dynablock FPs on legit msg submission

> ------- Additional Comments From jm@jmason.org  2004-12-15 13:15 -------
> I think the received-parser test suite was not updated.  I'll take a look.
> FWIW, if it's just the addition of "auth=" to most of the patterns, and it
> doesn't throw up a bug, I'll just use the existing +1's ;)

That is probably it, I'll let you fix it up.  Consider my +1 renewed
when you get the tests updated.

Michael

ok -- applied! r112024 in trunk, r112026 in b3_0.

as expected, the test failures were due to the addition of the "auth=" string. 
I also took the opportunity to throw in many of the test cases from this bug...

Oops, didn't think to make sure it didn't break the test script.  Thanks for
fixing that Justin.

Daryl

Hi,

After puzzeling over this for a while I was going to raise a bug,
However luckily I found this first and found the solution (which is to make the 
with SMTP to with ESMTPA as RFC 3848).
However before doing this I read 
http://wiki.apache.org/spamassassin/DynablockIssues
Which now I know the solution it does mention however not very clearly.
It does also suggest another solution to add a LOCAL_AUTH_RCVD rule, which I 
tried however this doesn't work.  The rule isn't anywhere in the SA code and 
doesn't counteract the block list issue.  I did manage to get it to work by 
giving the rule a negative score to counteract the otehr rules but this isn't 
very nice.
Would it be possible to update the wiki page to explain about RFC 3848 briefly, 
and also to either removed the LOCAL_AUTH_RCVD stuff or make it work?

Regards
Gary

The wiki page describes a rule you can add yourself if support for your MTAs
authentication tokens isn't already in SpamAssassin.  You'd need to add the
'header' rule described along with a 'score' and 'describe' line to go with it.

If you're using Postfix you're out of luck though.  Postfix isn't compliant with
RFC 3848.  If this is the case, I'd suggest you ask the Wietse Venema
(www.postfix.org) to add RFC 3848 support to Postfix.  A cursory glance at
Postfix's code reveals that compliance would be trivial.

You'll have to point out to him that RFC 3848 compliance wouldn't reveal HOW a
user authenticated, something that he has spoken out against doing.

The SMTP AUTH patch for qmail [1] recently added RFC 3848 support. The received
header looks now like this:

Received: (qmail 25253 invoked by uid 90); 10 Mar 2005 16:48:49 -0000
Received: from unknown (HELO ?192.168.1.20?) (webmaster@myhosting.de@80.184.166.249)
  by 0 with ESMTPA; 10 Mar 2005 16:48:49 -0000

Because of the semicolon after ESMTPA the regexp doesn't match, but if I correct
the regexp the mail doesn't get detected as smtp auth either. That is because of
$inferred_as_trusted gets set in Received.pm (beginning line 159) (don't know
why exactly).

I now avoid these checks by setting a value for trusted_networks, but that's
merely a workaround.

[1] http://www.fehcom.de/qmail/smtpauth.html

Subject: Re:  [review] detect SMTP AUTH to avoid dynablock FPs on
 legit msg submission

Could you please attach a sample message to this bug (not pasted in the 
comment text)?

Thanks.

Better would probably be to open up a seperate bug and track it that way.