|Summary:||SPF_PASS not relieable|
|Product:||Spamassassin||Reporter:||Reindl Harald <h.reindl>|
|Component:||Plugins||Assignee:||SpamAssassin Developer Mailing List <dev>|
|Severity:||normal||CC:||apache, davej, h.reindl, rwmaillists, wolfsplat|
Description Reindl Harald 2016-09-01 10:21:07 UTC
as far as i know SA is supposed to re-use the SPF-header vom spf-policyd well, the message below has a clear spf-pass but no SPF hits in the report header and hence "whitelist_auth" did also not trigger even if it would not re-use the "Received-SPF" header - since it was inserted by the MTA before passing the message to the milter it is 100% sure in the local resolver cache and dns-timeouts or errors are practically not possible at that stage feels somehow like a gambling machine and "whitelist_auth" needs to be 100% relieable (not for mailchimp like in this case but in general to distinct between forged fincancial mails and real ones) __________________________________________ Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=188.8.131.52; helo=mail53.suw15.mcsv.net; firstname.lastname@example.org; email@example.com __________________________________________ -Spam-Report: Flag: No, * -0.2 CUST_DNSWL_8_TL_N RBL: dnswl-aggregate.thelounge.net (No Trust) * [184.108.40.206 listed in dnswl-aggregate.thelounge.net] * -0.4 RCVD_IN_MSPIKE_H5 RBL: Excellent reputation (+5) * [220.127.116.11 listed in wl.mailspike.net] * 0.3 URIBL_GREY Contains an URL listed in the URIBL greylist * [URIs: campaign-archive2.com] * 1.0 NIXSPAM_IXHASH DIGEST: ix.dnsbl.manitu.net * -0.1 CUST_DNSWL_5_ORG_N RBL: list.dnswl.org (No Trust) * [18.104.22.168 listed in list.dnswl.org] * 0.1 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail * domains are different * -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain * 0.5 CUST_BODY_BEGINS_VL BODY: Begins Very Low * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.5 BAYES_50 BODY: Bayes spam probability is 40 to 60% * [score: 0.5000] * 0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's * domain * 1.5 IXHASH_CHECK Message hits one ore more IXHASH digest-sources * -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders * 0.0 T_OBFU_ATTACH_MISSP No description available. * 0.1 BOGOFILTER_UNSURE BOGOFILTER: message is Unsure with * bogofilter-score 0.5004 X-Virus-Scanned: Yes
Comment 1 Reindl Harald 2016-09-01 10:22:33 UTC
well, and when i take that message and feed it to spamassassin now without any config changes as expected whitelist_auth triggers: Content analysis details: (-100.7 points, 5.5 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.2 CUST_DNSWL_8_TL_N RBL: dnswl-aggregate.thelounge.net (No Trust) [22.214.171.124 listed in dnswl-aggregate.thelounge.net] -0.1 CUST_DNSWL_5_ORG_N RBL: list.dnswl.org (No Trust) [126.96.36.199 listed in list.dnswl.org] -0.4 RCVD_IN_MSPIKE_H5 RBL: Excellent reputation (+5) [188.8.131.52 listed in wl.mailspike.net] -100 USER_IN_SPF_WHITELIST From: address is in the user's SPF whitelist 0.0 SHORTCIRCUIT Not all rules were run, due to a shortcircuited rule 0.0 CUST_SHORTCIRCUIT Skip tests based on whitelists/blacklists and local relays
Comment 2 AXB 2016-09-01 10:29:25 UTC
(In reply to Reindl Harald from comment #0) > as far as i know SA is supposed to re-use the SPF-header vom spf-policyd Where is this documented?
Comment 3 Reindl Harald 2016-09-01 10:37:37 UTC
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_SPF.html ignore_received_spf_header (0|1) (default: 0) By default, to avoid unnecessary DNS lookups, the plugin will try to use the SPF results found in any Received-SPF headers it finds in the message that could only have been added by an internal relay. Set this option to 1 to ignore any Received-SPF headers present and to have the plugin perform the SPF check itself. Note that unless the plugin finds an identity=helo, or some unsupported identity, it will assume that the result is a mfrom SPF check result. The only identities supported are mfrom, mailfrom and helo.
Comment 4 RW 2016-09-01 11:34:52 UTC
For this to work the Received-SPF header needs to be above a trusted Received header.
Comment 5 RW 2016-09-01 11:38:52 UTC
It shouldn't prevent SPF working though. It should fall-back to doing its own lookup.
Comment 6 Reindl Harald 2016-09-01 11:44:44 UTC
it is above a trusted-received header, in fact it's on top of the message when handover to the milter happens and if that received header would not be there and trusted all the DNSBL won't work too "It shouldn't prevent SPF working though. It should fall-back to doing its own lookup" is what i expressed with "even if it would not re-use the "Received-SPF" header - since it was inserted by the MTA before passing the message to the milter it is 100% sure in the local resolver cache and dns-timeouts or errors are practically not possible at that stage" so there are two anchors which should make SPF_PASS successful: * the fact that's the response is in the DNS cache * the fact of the poliycd-header with made sure it's in the DNS cache since policyd is running long before milters _________________________________ Name: mail-gw.thelounge.net Address: 10.0.0.19 clear_trusted_networks trusted_networks 10.0.0.0/24 internal_networks 10.0.0.0/24 Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=184.108.40.206; helo=mail53.suw15.mcsv.net; firstname.lastname@example.org uw15.mcsv.net; email@example.com Received: from mail53.suw15.mcsv.net (mail53.suw15.mcsv.net [220.127.116.11]) by mail-gw.thelounge.net (THELOUNGE GATEWAY) with ESMTP id 3sPYXJ39pvz9c9 for <firstname.lastname@example.org>; Wed, 31 Aug 2016 20:17:52 +0200 (CEST)
Comment 7 Reindl Harald 2016-10-01 12:14:41 UTC
is there any news? the header is pretty clear and for sure present in the milter because the spf-policyd is running long before Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=18.104.22.168; helo=a5-3.smtp-out.eu-west-1.amazonses.com; envelope-from=010201578022b73d-db512ddc-95e7-4730-bd72-e1fb9fb0bbdf-000000@ mailer.netflix.com another case where SPF_PASS is missing and hence "whitelist_auth *@mailer.netflix.com" not triggered but HTTPS_HTTP_MISMATCH because the so not happening shortcircuit in that case BAYES_00 and no problem, but god beware this happens for a often forged messagetype resulting in BAYES_99 combined with other rules which is the reason you used whitelist_auth
Comment 8 Dave Jones 2018-01-28 16:43:10 UTC
If this is still a problem, this should be fixed sooner rather than later. I have not run across a time when SPF_PASS didn't match my own Authentication-Results header being added by OpenDMARC and python-policyd-spf.
Comment 9 Reindl Harald 2018-01-28 16:51:10 UTC
well, how should that got fixed when there was no SA release for almost two years
Comment 10 Dave Jones 2018-01-28 17:02:14 UTC
Perhaps Authentication-Results is handled differently than Received-SPF. SA 3.4.2 is coming out soon so maybe I am trying to help get attention to this issue to get it into 3.4.2 for you.