Bug 7540 - r1822466 fails lint check (FORGED_GMAIL_RCVD)
Summary: r1822466 fails lint check (FORGED_GMAIL_RCVD)
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Plugins (show other bugs)
Version: 3.4.1
Hardware: PC FreeBSD
: P2 major
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
: 7542 7543 7544 (view as bug list)
Depends on: 6946
Blocks:
  Show dependency tree
 
Reported: 2018-01-30 11:24 UTC by Jeremy Chadwick
Modified: 2019-01-10 18:10 UTC (History)
16 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
Enable rule only on 3.4.2+ patch None Giovanni Bechis [HasCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Jeremy Chadwick 2018-01-30 11:24:59 UTC
This early morning of 2018/01/30 our SpamAssassin 3.4.1 systems ran sa-update, which picked up the latest updates from 1822617.tar.gz (presumably correlates with r1822617 in SVN).  sa-update failed:
 
rules: failed to run FORGED_GMAIL_RCVD test, skipping:
        (Can't locate object method "check_for_forged_gmail_received_headers" via package "Mail::SpamAssassin::PerMsgStatus" at (eval 1313) line 1111.)

This is the responsible commit, related to Bug 6946 --

https://svn.apache.org/viewvc?view=revision&revision=1822466
Comment 1 Giovanni Bechis 2018-01-30 11:29:13 UTC
Created attachment 5519 [details]
Enable rule only on 3.4.2+

This rule was intended only for 3.4.2+ versions, fix attached.
Comment 2 Dave Jones 2018-01-30 13:14:05 UTC
Patch applied to check for version 3.4.2.  Running test harness now and will commit shortly.
Comment 3 Dave Jones 2018-01-30 14:16:56 UTC
$ svn commit rules/20_head_tests.cf -m "Bug 7540"
Sending        rules/20_head_tests.cf
Transmitting file data .done
Committing transaction...
Committed revision 1822648.
Comment 4 marco.aimar 2018-01-31 08:08:54 UTC
Hi, we have same problem. How I can see when patch will be published and available with "sa-update"?
Thank you
Comment 5 deeend 2018-01-31 09:14:07 UTC
Hi All, 

Although an update 1822773 has been published the problem still persists upon trying to update SpamAssassin:

sa-update -v
Update available for channel updates.spamassassin.org: 1822491 -> 1822773
http: (curl) GET http://sa-update.ena.com/1822773.tar.gz, success
http: (curl) GET http://sa-update.ena.com/1822773.tar.gz.sha1, success
http: (curl) GET http://sa-update.ena.com/1822773.tar.gz.asc, success
rules: failed to run FORGED_GMAIL_RCVD test, skipping:
        (Can't locate object method "check_for_forged_gmail_received_headers" via package "Mail::SpamAssassin::PerMsgStatus" at (eval 1339) line 123, <GEN13> line 1.
)
channel: lint check of update failed, channel failed
Update failed, exiting with code 4

Regards
Comment 6 Mike Bird 2018-01-31 10:23:23 UTC
Problem still exists.

I see your patch via ViewVC in 1822773 but the patch is not in the 20_head_tests.cf in the 1822773.tar.gz that sa-update retrieves from the updates.spamassassin.org channel.
Comment 7 Kevin A. McGrail 2018-01-31 12:14:31 UTC
The problem will exist until the system builds a new tar file which is a timing issue.  Depending on the checkin and the masscheck system which runs on the submissions from our masscheckers, rules may take time to publish.  As this fails lint before installation, we are not forcing an update.  The issue is resolved and expect it to pass lint with a completely new rules file when published.

The old rule file is not replaced.  It's considered a release artifact.

HTH, KAM
Comment 8 Kevin A. McGrail 2018-01-31 12:31:07 UTC
*** Bug 7542 has been marked as a duplicate of this bug. ***
Comment 9 Dave Jones 2018-01-31 14:54:23 UTC
It can take up to 48 hours for a rule update commit to make it through the masscheck process and into the sa-update tgz file.  This should be resolved in the morning/tomorrow.
Comment 10 Kevin A. McGrail 2018-01-31 15:04:41 UTC
I caution that it can take longer.  For example, if the masscheck submissions aren't sufficient to fulfill the threshold set for an effective update.
Comment 11 Kevin A. McGrail 2018-01-31 15:05:32 UTC
*** Bug 7543 has been marked as a duplicate of this bug. ***
Comment 12 Robert R. Richter 2018-01-31 16:51:23 UTC
Should we still wait for the update, or are any actions required to get it working again?
Comment 13 Dave Jones 2018-01-31 17:01:56 UTC
No special actions needed.  sa-update will should work tomorrow just fine.  If you have sa-update being run via cron or by some other automatic means then you are good to go.
Comment 14 Kevin A. McGrail 2018-01-31 21:56:24 UTC
*** Bug 7544 has been marked as a duplicate of this bug. ***
Comment 15 ruga 2019-01-02 15:17:36 UTC
>sa-update -V                                                                                                                                                                       
sa-update version 3.4.2 / svn1840377
  running on Perl version 5.28.1

>sa-update
[...]
Jan  2 15:32:02.462 [42029] dbg: FreeMail: RULE (__smf_freemail_hdr_replyto) check_freemail_header
Jan  2 15:32:02.462 [42029] dbg: FreeMail: header Reply-To:addr not found from mail
rules: failed to run FORGED_GMAIL_RCVD test, skipping:
        (Can't locate object method "check_for_forged_gmail_received_headers" via package "Mail::SpamAssassin::PerMsgStatus" at (eval 1409) line 803.
)
Jan  2 15:32:02.463 [42029] dbg: FreeMail: RULE (__freemail_hdr_replyto) check_freemail_header
[...]
Comment 16 Giovanni Bechis 2019-01-03 18:56:37 UTC
I cannot reproduce, could you post a sample email that triggers the problem and a full "spamassassin -D all <email" run ?
Comment 17 ruga 2019-01-05 11:55:09 UTC
(In reply to Giovanni Bechis from comment #16)
> I cannot reproduce, could you post a sample email that triggers the problem
> and a full "spamassassin -D all <email" run ?

The problem is not triggered by an email.

I was just upgrading SA, and got stuck with the above.
Comment 18 ruga 2019-01-09 18:47:28 UTC
>sudo sa-update -v -D 4 --refreshmirrors

Update available for channel updates.spamassassin.org: -1 -> 1850717
http: (curl) GET http://spamassassin.apache.org/updates/MIRRORED.BY, success
http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz, success
http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.sha512, success
http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.sha256, success
http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.asc, success
rules: failed to run FORGED_GMAIL_RCVD test, skipping:
	(Can't locate object method "check_for_forged_gmail_received_headers" via package "Mail::SpamAssassin::PerMsgStatus" at (eval 1419) line 887.
)
channel: lint check of update failed, channel failed
Update failed, exiting with code 4
Comment 19 Bill Cole 2019-01-09 20:14:12 UTC
(In reply to ruga from comment #18)
> >sudo sa-update -v -D 4 --refreshmirrors

The '4' you use in that comnmand is not meaningful. Any argument to the "-D" option must be a comma-delimited list of debug channel names for SA or the word "all". 

 
> Update available for channel updates.spamassassin.org: -1 -> 1850717
> http: (curl) GET http://spamassassin.apache.org/updates/MIRRORED.BY, success
> http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz, success
> http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.sha512, success
> http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.sha256, success
> http: (curl) GET http://sa-update.secnap.net/1850717.tar.gz.asc, success
> rules: failed to run FORGED_GMAIL_RCVD test, skipping:
> 	(Can't locate object method "check_for_forged_gmail_received_headers" via
> package "Mail::SpamAssassin::PerMsgStatus" at (eval 1419) line 887.
> )

The method check_for_forged_gmail_received_headers() is implemented in Mail::SpamAssassin::Plugin::HeaderEval which the error message above implies is not loaded. Mail::SpamAssassin::Plugin::HeaderEval is loaded by default in all versions of SA supported by the default rules channel by this line in the distributed version of v320.pre, located in the site configuration directory (typically /etc/mail/spamassassin):

loadplugin Mail::SpamAssassin::Plugin::HeaderEval

If you have commented out that line or deleted v320.pre or explicitly unloaded the HeaderEval plugin in some other configuration file, the default rules channel will not work.
Comment 20 ruga 2019-01-10 08:56:54 UTC
Re: 4
------
errata corrige: "4" -> "-4"

Re: v320.pre
-------------
The module is loaded:

>grep HeaderEval /etc/mail/spamassassin/v320.pre
loadplugin Mail::SpamAssassin::Plugin::HeaderEval

> sudo sa-update -v -D -4 --refreshmirrors 2>&1 | grep HeaderEval                        
Jan 10 09:55:11.733 [72854] dbg: plugin: loading Mail::SpamAssassin::Plugin::HeaderEval from @INC
Jan 10 09:55:15.659 [72854] dbg: plugin: loading Mail::SpamAssassin::Plugin::HeaderEval from @INC
Comment 21 ruga 2019-01-10 11:25:24 UTC
>sudo zgrep -r check_for_forged_gmail_received_headers /var/spamd
Binary file /var/spamd/3.004002/updates_spamassassin_org/1850825.tar.gz matches

>mkdir /tmp/spamd
>tar -xf /var/spamd/3.004002/updates_spamassassin_org/1850825.tar.gz -C /tmp/spamd
>grep -r check_for_forged_gmail_received_headers /tmp/spamd
/tmp/spamd/20_head_tests.cf:header FORGED_GMAIL_RCVD	eval:check_for_forged_gmail_received_headers()

OK

>sudo sa-update -v -D -4 --refreshmirrors 2>&1 | grep -A3 -B3 check_for_forged_gmail_received_headers 
Jan 10 12:11:27.142 [94699] dbg: config: header eval rule name is FORGED_HOTMAIL_RCVD2 function is check_for_no_hotmail_received_headers()
Jan 10 12:11:27.142 [94699] dbg: config: header eval rule name is __FORGED_EUDORAMAIL_RCVD function is check_for_forged_eudoramail_received_headers()
Jan 10 12:11:27.142 [94699] dbg: config: header eval rule name is FORGED_YAHOO_RCVD function is check_for_forged_yahoo_received_headers()
Jan 10 12:11:27.142 [94699] dbg: config: header eval rule name is FORGED_GMAIL_RCVD function is check_for_forged_gmail_received_headers()
^^^ OK
Jan 10 12:11:27.143 [94699] dbg: config: header eval rule name is __FORGED_JUNO_RCVD function is check_for_forged_juno_received_headers()
Jan 10 12:11:27.143 [94699] dbg: config: header eval rule name is SORTED_RECIPS function is sorted_recipients()
Jan 10 12:11:27.143 [94699] dbg: config: header eval rule name is SUSPICIOUS_RECIPS function is similar_recipients('0.65','undef')
--
--
Jan 10 12:11:28.577 [94699] dbg: FreeMail: RULE (FREEMAIL_ENVFROM_END_DIGIT) check_freemail_header regex:\d@
Jan 10 12:11:28.577 [94699] dbg: FreeMail: header EnvelopeFrom not found from mail
rules: failed to run FORGED_GMAIL_RCVD test, skipping:
	(Can't locate object method "check_for_forged_gmail_received_headers" via package "Mail::SpamAssassin::PerMsgStatus" at (eval 1427) line 467.
)
Jan 10 12:11:28.578 [94699] dbg: FreeMail: RULE (FREEMAIL_REPLYTO_END_DIGIT) check_freemail_header regex:\d@
Jan 10 12:11:28.578 [94699] dbg: FreeMail: header Reply-To not found from mail
Comment 22 ruga 2019-01-10 11:36:16 UTC
@Bill Cole

>less 20_head_tests.cf 

[...]
ifplugin Mail::SpamAssassin::Plugin::HeaderEval
[...]
if (version >= 3.004002)
header FORGED_GMAIL_RCVD        eval:check_for_forged_gmail_received_headers()
describe FORGED_GMAIL_RCVD      'From' gmail.com does not match 'Received' headers
endif
[...]
endif

So, check_for_forged_gmail_received_headers is called, because Mail::SpamAssassin::Plugin::HeaderEva was loaded and version >= 3.004002.
Comment 23 ruga 2019-01-10 11:47:47 UTC
@Bill Cole

The function "check_for_forged_gmail_received_headers" is not defined: 

>ls -l /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/Plugin/HeaderEval.pm
-r--r--r--  1 root  wheel  39279 Jan  2 14:54 /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/Plugin/HeaderEval.pm

>grep check_for_forged_gmail_received_headers /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/Plugin/HeaderEval.pm 
no answer

Also, 

>grep -r check_for_forged_gmail_received_headers /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/  
no answer

The function is missing from the distribution:

> grep -r check_for_forged_gmail_received_headers /opt/src/Mail-SpamAssassin-3.4.2
no answer

So, after one year, the function is still not being distributed.
Comment 24 Giovanni Bechis 2019-01-10 12:06:42 UTC
Freshly donwloaded 3.4.2 from https://www.apache.org/dist/spamassassin/source/

$ md5 lib/Mail/SpamAssassin/Plugin/HeaderEval.pm                                                                                 
MD5 (lib/Mail/SpamAssassin/Plugin/HeaderEval.pm) = 63c440f27bf47dd87a265aa07a67d063

Do you have the same hash of this file ?
Comment 25 ruga 2019-01-10 12:08:30 UTC
My md5 returns 125ff7007ad16415522a3c6665c2abff.
Comment 26 ruga 2019-01-10 12:09:16 UTC
Downloaded from http://www.eu.apache.org/dist/spamassassin/source
Comment 27 ruga 2019-01-10 12:11:30 UTC
>ls -l /opt/src/Mail-SpamAssassin-3.4.2.*

-rw-rw-rw-  1 rg  staff  2700016 Sep 15 08:24 /opt/src/Mail-SpamAssassin-3.4.2.tar.bz2
-rw-rw-rw-  1 rg  staff      162 Sep 15 08:24 /opt/src/Mail-SpamAssassin-3.4.2.tar.bz2.sha512

>cat /opt/src/Mail-SpamAssassin-3.4.2.tar.bz2.sha512                                                                                                                                
fe3d9d1d7b9fed3063549afd071066729f1f4d998be91ded1e5afc29bb37c7a298dc5f8f99a282b75435d317b5b5072a81393134ccfe059a73d953e26a9c3885  Mail-SpamAssassin-3.4.2.tar.bz2
Comment 28 Bill Cole 2019-01-10 14:04:33 UTC
(In reply to Giovanni Bechis from comment #24)
> Freshly donwloaded 3.4.2 from
> https://www.apache.org/dist/spamassassin/source/
> 
> $ md5 lib/Mail/SpamAssassin/Plugin/HeaderEval.pm                            
> 
> MD5 (lib/Mail/SpamAssassin/Plugin/HeaderEval.pm) =
> 63c440f27bf47dd87a265aa07a67d063
> 
> Do you have the same hash of this file ?

That matches the md5 of HeaderEval.pm installed from the distributed package, in the original source from SVN, in the 'bliub'm version built from SVN, and in the copy I just extracted from a freshly downloaded distribution tarball. 

check_for_forged_gmail_received_headers is defined on line 584 of the file with that hash.

There is something wrong with your installation that I cannot reproduce. You may find it useful to take this to the SpamAssassin Users mailing list, where a broader range of imagination is available to help guess what the problem is. 

spamassassin_release_3_4_2# svn status -u
Status against revision: 1850937
spamassassin_release_3_4_2# find ./ /opt/local/lib/perl5 -type f -name HeaderEval.pm -exec md5 -r {} \;
63c440f27bf47dd87a265aa07a67d063 .//blib/lib/Mail/SpamAssassin/Plugin/HeaderEval.pm
63c440f27bf47dd87a265aa07a67d063 .//lib/Mail/SpamAssassin/Plugin/HeaderEval.pm
63c440f27bf47dd87a265aa07a67d063 /opt/local/lib/perl5/vendor_perl/5.26/Mail/SpamAssassin/Plugin/HeaderEval.pm

spamassassin_release_3_4_2# find ./ /opt/local/lib/perl5 -type f -name HeaderEval.pm -exec grep -Hn '^sub check_for_forged_gmail_received_headers' {} \;
.//blib/lib/Mail/SpamAssassin/Plugin/HeaderEval.pm:584:sub check_for_forged_gmail_received_headers {
.//lib/Mail/SpamAssassin/Plugin/HeaderEval.pm:584:sub check_for_forged_gmail_received_headers {
/opt/local/lib/perl5/vendor_perl/5.26/Mail/SpamAssassin/Plugin/HeaderEval.pm:584:sub check_for_forged_gmail_received_headers {

Downloads$ shasum -a 512 Mail-SpamAssassin-3.4.2.tar.bz2
fe3d9d1d7b9fed3063549afd071066729f1f4d998be91ded1e5afc29bb37c7a298dc5f8f99a282b75435d317b5b5072a81393134ccfe059a73d953e26a9c3885  Mail-SpamAssassin-3.4.2.tar.bz2

Downloads$ tar xf Mail-SpamAssassin-3.4.2.tar.bz2  */HeaderEval.pm

Downloads$ md5 -r Mail-SpamAssassin-3.4.2/lib/Mail/SpamAssassin/Plugin/HeaderEval.pm 
63c440f27bf47dd87a265aa07a67d063 Mail-SpamAssassin-3.4.2/lib/Mail/SpamAssassin/Plugin/HeaderEval.pm
Comment 29 Bill Cole 2019-01-10 18:10:16 UTC
(In reply to Bill Cole from comment #28)

> package, in the original source from SVN, in the 'bliub'm version built from

s/'bliub'm/'blib'/ 

(The MakeMaker pre-install staging directory)

==============================================================================

(In reply to ruga from comment #23)
> @Bill Cole
> 
> The function "check_for_forged_gmail_received_headers" is not defined: 
> 
> >ls -l /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/Plugin/HeaderEval.pm
> -r--r--r--  1 root  wheel  39279 Jan  2 14:54
> /usr/share/perl/lib/site_perl/5.28.1/Mail/SpamAssassin/Plugin/HeaderEval.pm

That is about 5k larger than the version of that file in the v3.4.2 release. Your installation is definitely broken.