Bug 7886 - Qmail received header fails when FcrDNS is ok and TLS is used ( ESMTPS )
Summary: Qmail received header fails when FcrDNS is ok and TLS is used ( ESMTPS )
Status: NEW
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Libraries (show other bugs)
Version: 3.4.4
Hardware: PC Mac OS X
: P2 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-05 11:58 UTC by Jose Borges Ferreira
Modified: 2021-03-07 19:17 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
patch patch None Jose Borges Ferreira [NoCLA]
sample email for testing message/rfc822 None Jose Borges Ferreira [NoCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Jose Borges Ferreira 2021-03-05 11:58:51 UTC
Created attachment 5739 [details]
patch

When receiving an email with FcrDNS and TLS qmail creates an header like this:

Received: from mta.example.net ([192.0.0.3])
          (envelope-sender <esmtps@example.net>)
          by qmail-mta02 (qmail-qmail-1.0.0) with ESMTPS
          for <user@example.net>; 3 Mar 2021 17:42:22 -0000

which translates to

Mar  5 11:41:57.086 [59009] dbg: received-header: parsed as [ ip=192.0.0.3 rdns= helo=mta.example.net by=qmail-mta02 ident= envfrom=esmtps@example.net intl=0 id= auth= msa=0 ]

with "rdns= " and triggers RDNS_NONE and then others .
The attached sample can be used to test

cat sample.eml | spamassassin  -Dreceived-header  2>&1 | grep parsed

Before patch :

Mar  5 11:41:57.085 [59009] dbg: received-header: parsed as [ ip=192.0.0.2 rdns=mta.example.net helo=mta.example.net by=qmail-mta02 ident= envfrom=smtp@example.net intl=0 id= auth= msa=0 ]
Mar  5 11:41:57.086 [59009] dbg: received-header: parsed as [ ip=192.0.0.3 rdns= helo=mta.example.net by=qmail-mta02 ident= envfrom=esmtps@example.net intl=0 id= auth= msa=0 ]
Mar  5 11:41:57.086 [59009] dbg: received-header: parsed as [ ip=192.0.0.1 rdns=mta.example.com helo=helo.example.com by=qmail-mta01 ident= envfrom=esmtps@example.com intl=0 id= auth= msa=0 ]
Mar  5 11:41:57.087 [59009] dbg: received-header: parsed as [ ip=192.0.0.4 rdns=mta.example.com helo=helo.example.com by=qmail-mta01 ident= envfrom=smtp@example.com intl=0 id= auth= msa=0 ]

After patch :
Mar  5 11:41:31.939 [58932] dbg: received-header: parsed as [ ip=192.0.0.2 rdns=mta.example.net helo=mta.example.net by=qmail-mta02 ident= envfrom=smtp@example.net intl=0 id= auth= msa=0 ]
Mar  5 11:41:31.939 [58932] dbg: received-header: parsed as [ ip=192.0.0.3 rdns=mta.example.net helo=mta.example.net by=qmail-mta02 ident= envfrom=esmtps@example.net intl=0 id= auth= msa=0 ]
Mar  5 11:41:31.940 [58932] dbg: received-header: parsed as [ ip=192.0.0.1 rdns=mta.example.com helo=helo.example.com by=qmail-mta01 ident= envfrom=esmtps@example.com intl=0 id= auth= msa=0 ]
Mar  5 11:41:31.940 [58932] dbg: received-header: parsed as [ ip=192.0.0.4 rdns=mta.example.com helo=helo.example.com by=qmail-mta01 ident= envfrom=smtp@example.com intl=0 id= auth= msa=0 ]
Comment 1 Jose Borges Ferreira 2021-03-05 11:59:25 UTC
Created attachment 5740 [details]
sample email for testing
Comment 2 RW 2021-03-05 15:59:31 UTC
Not being familiar with qmail I found this a little confusing. mta.example.net is in the correct position for rDNS in qmail. The issue is that this qmail format wont be used unless the header matches a regex ending in 

...with (.* )?(SMTP|QMQP)(?! id )

so it doesn't match ESMTPS. 

The reason for only having SMTP is given in a comment as being to avoid matching on "with ESMTP" from sendmail, but  (.* )? will match on the E, so that's never worked anyway.


"(?! id )" was added later for: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=4813
Comment 3 Jose Celestino 2021-03-06 12:32:23 UTC
(In reply to RW from comment #2)
> 
> The reason for only having SMTP is given in a comment as being to avoid
> matching on "with ESMTP" from sendmail, but  (.* )? will match on the E, so
> that's never worked anyway.

Some qmail installations support RFC 3848, so the comment is no longer valid and you can expect to see "ESMTP[S|A|SA]" on qmail Received headers.

Don't see how (.* )? will match on the E though (because of the trailing space, it's there to match the possible "cypher encrypted " prefixing the protocol).
Comment 4 RW 2021-03-06 14:50:07 UTC
(In reply to Jose Celestino from comment #3)

> Don't see how (.* )? will match on the E though (because of the trailing
> space, it's there to match the possible "cypher encrypted " prefixing the
> protocol).

It would match on the E in "with ESMTP " from Sendmail thereby removing any benefit of having SMTP rather than E?SMTP at the time it was written. This is important because it means your change is not removing a long-standing mitigation.

I think the "(?! id )" check, from bug 4813, probably removed any confusion between Qmail and Sendmail. There would have been a lengthy period for remaining problems to be spotted between Bug 4813 and ESMTPS becoming the norm.

It's important to get it right though because the converse problem of Sendmail headers being parsed as qmail could lead to spam being whitelisted based on a forged helo rather than FcrDNS.
Comment 5 Jose Celestino 2021-03-06 23:35:43 UTC
(In reply to RW from comment #4)

> It would match on the E in "with ESMTP " from Sendmail thereby removing any
> benefit of having SMTP rather than E?SMTP at the time it was written. This
> is important because it means your change is not removing a long-standing
> mitigation.

It would match on "with E SMTP", not "with ESMTP", or maybe I'm missing something?
Comment 6 RW 2021-03-07 19:17:41 UTC
(In reply to Jose Celestino from comment #5)
> (In reply to RW from comment #4)
> 
> > It would match on the E in "with ESMTP " from Sendmail thereby removing any
> > benefit of having SMTP rather than E?SMTP at the time it was written. This
> > is important because it means your change is not removing a long-standing
> > mitigation.
> 
> It would match on "with E SMTP", not "with ESMTP", or maybe I'm missing
> something?

No, you are right, I didn't notice that space. When you referred to a trailing space I thought you meant after SMTP. 

Perhaps someone who knows Sendmail could comment on whether  "(?! id )" will eliminate Sendmail matches reliably.