Bug 7838 - sa-update failes with error message (warning)
Summary: sa-update failes with error message (warning)
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 3.4.2
Hardware: PC Linux
: P2 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
: 7839 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-07-12 19:35 UTC by Lars D.
Modified: 2020-07-20 01:45 UTC (History)
6 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Lars D. 2020-07-12 19:35:26 UTC
The rule update with sa-update failed this morning with the following error:

config: warning: description exists for non-existent rule USER_IN_ALLOWLIST_TO

channel: lint check of update failed, channel failed
sa-update failed for unknown reasons

Although it‘s a warning, the update failed.

Sa-update has downloaded the following files:
1879784.tar.gz
1879784.tar.gz.asc
1879784.tar.gz.sha256
1879784.tar.gz.sha512
MIRRORED.BY

I unpacked the archive and did a grep for string USER_IN_ALLOWLIST_TO:

grep -a USER_IN_ALLOWLIST_TO *
30_text_de.cf:lang de describe USER_IN_ALLOWLIST_TO Empf�ngeradresse steht in Ihrer pers�nlichen wei�en Liste
30_text_fr.cf:lang fr describe USER_IN_ALLOWLIST_TODestinataire sur la liste blanche (config SA)
30_text_pl.cf:lang pl describe USER_IN_ALLOWLIST_TOU�ytkownik jest wymieniony w 'whitelist_to'

My system is: Debian GNU/Linux 10 (buster)
Comment 1 Kevin A. McGrail 2020-07-12 20:52:19 UTC
Thanks.  Just a warning you can ignore.  I've submitted a fix for 3 descriptions missed in changing ALLOWLIST_TO to WELCOMELIST_TO.

The next rule publishing will resolve it.

svn commit -m 'fixed a few descriptions for ALLOWLIST_TO to WELCOMELIST_TO' 
Sending        rules/30_text_de.cf
Sending        rules/30_text_fr.cf
Sending        rules/30_text_pl.cf
Transmitting file data ...
Committed revision 1879809.
Comment 2 John Hardin 2020-07-13 16:58:33 UTC
*** Bug 7839 has been marked as a duplicate of this bug. ***
Comment 3 greinick 2020-07-15 18:22:03 UTC
Hi, I came across that Bug ticket, while searching for the message popping up when we download the updates:

sa-update --nogpg --channel updates.spamassassin.org -v

Update available for channel updates.spamassassin.org: 1879434 -> 1879817
http: (curl) GET http://www.sa-update.pccc.com/1879817.tar.gz, success
http: (curl) GET http://www.sa-update.pccc.com/1879817.tar.gz.sha512, success
config: warning: description exists for non-existent rule USER_IN_WELCOMELIST_TO

channel: lint check of update failed, channel failed
Update failed, exiting with code 4


Unpacking the archive gripping for USER_IN_WELCOMELIST_TO:

30_text_de.cf:lang de describe USER_IN_WELCOMELIST_TO Empf?ngeradresse steht in Ihrer pers?nlichen wei?en Liste
30_text_fr.cf:lang fr describe USER_IN_WELCOMELIST_TO	Destinataire sur la liste blanche (config SA)
30_text_pl.cf:lang pl describe USER_IN_WELCOMELIST_TO	U?ytkownik jest wymieniony w 'whitelist_to'
30_text_pt_br.cf:lang pt_BR describe USER_IN_WELCOMELIST_TO Usu?rio est? listado na 'whitelist_to'


I guess the "umlaute" are encoded wrong. Other channels do also provide files with umlauts and don't break. 

Or it s something different?

CentOS Linux release 7.7.1908
Comment 4 Kevin A. McGrail 2020-07-15 20:38:57 UTC
It's a different issue where we added a functionality check for a rule and descriptions for two scenarios.  SA 3.4.2 and older will not allow this for sa-update so it fails.  I've submitted a patch for this to get processed by masscheck and hopefully released in the next 24-48 hours is my estimate.
Comment 5 Kevin A. McGrail 2020-07-16 19:11:05 UTC
I believe ruleset 1879934 has been published and should fix the issue.  Can anyone confirm, please?
Comment 6 fops 2020-07-17 05:16:47 UTC
issued the update right now.... =(

root@mx2:~# sa-update --nogpg --channel updates.spamassassin.org -v
Update available for channel updates.spamassassin.org: 1879434 -> 1879851
http: (curl) GET http://sa-update.dnswl.org/1879851.tar.gz, success
http: (curl) GET http://sa-update.dnswl.org/1879851.tar.gz.sha512, success
http: (curl) GET http://sa-update.dnswl.org/1879851.tar.gz.sha256, success
config: warning: description exists for non-existent rule USER_IN_WELCOMELIST_TO

channel: lint check of update failed, channel failed
Update failed, exiting with code 4
Comment 7 harald ms 2020-07-17 08:47:39 UTC
Status: 	RESOLVED FIXED 
How this? When the mistake persists.
sa-update fails since many days and no one with the proper gpg key is capable of fixing this "USER_IN_ALLOWLIST_TO"!

The mistake is NOT fixed.
Comment 8 Lars D. 2020-07-17 11:09:48 UTC
The rule for USER_IN_WELCOMELIST_TO is surrounded by an if statement

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)

files:
50_scores.cf
60_whitelist.cf

If I add this if statement in the 30_text_xx.cf files arround the USER_IN_WELCOMELIST_TO rule description, the lint check does not give a warning and I expect the update process would not fail.

The problem with the warning of the lint check is, that (at least in debian) the update script fails if the return code of the check is not 0.
Maybe the distros should not fail on return code 4, but I don‘t know if there are other cases when return code 4 is set.
Comment 9 Kevin A. McGrail 2020-07-17 16:56:21 UTC
the issue is considered fixed in revision control.  The publishing of a new release or ruleset is important but not required for the closing of the bug.

if 60_whitelist.cf looks like this, it's not the version I checked in a week ago:

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
  #bz7826 renames whitelist to welcomelist
  header USER_IN_WELCOMELIST_TO     eval:check_to_in_welcomelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'welcomelist_to'
  tflags USER_IN_WELCOMELIST_TO     userconf nice noautolearn
else
  header USER_IN_WHITELIST_TO     eval:check_to_in_whitelist()
  describe USER_IN_WHITELIST_TO   User is listed in 'whitelist_to'
  tflags USER_IN_WHITELIST_TO     userconf nice noautolearn
endif

The correct version looks like this:

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
  #bz7826 renames whitelist to welcomelist
  header USER_IN_WELCOMELIST_TO     eval:check_to_in_welcomelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'welcomelist_to'
  tflags USER_IN_WELCOMELIST_TO     userconf nice noautolearn
else
  header USER_IN_WELCOMELIST_TO     eval:check_to_in_whitelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'whitelist_to'
  tflags USER_IN_WELCOMELIST_TO     userconf nice noautolearn
endif

I'll see if someone can put some eyeballs on why that isn't getting published.
Comment 10 hitd 2020-07-17 17:08:09 UTC
bug ist about USER_IN_ALLOWLIST_TO, not USER_IN_WELCOMELIST_TO.

And yes, updateprocess is still out of order here. So publishing a fixed ruleset is required for the closing of the bug.
Comment 11 harald ms 2020-07-17 17:32:02 UTC
Why the proper gpg key owner does not simply delete the lines in the explanation files for
french, german, polish, and portuguese-brazil?
rule USER_IN_ALLOWLIST_TO
?

I would have done it myself, but the sa-update complains about gpg-key of hash on the archive.

It appears to me like an April 1st joke.

rules/30_text_fr.cf etc.
they are not needed anyway, because for italian, spanish, greek, or any other languages there are none.
Comment 12 Lars D. 2020-07-17 17:34:34 UTC
My 60_whitelist.cf looks like this:

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
  #bz7826 renames whitelist to welcomelist
  header USER_IN_WELCOMELIST_TO     eval:check_to_in_welcomelist()
  describe USER_IN_WELCOMELIST_TO   User is listed in 'welcomelist_to'
  tflags USER_IN_WELCOMELIST_TO     userconf nice noautolearn                                                                                                                           else
  header USER_IN_WHITELIST_TO     eval:check_to_in_whitelist()
  describe USER_IN_WHITELIST_TO   User is listed in 'whitelist_to'
  tflags USER_IN_WHITELIST_TO     userconf nice noautolearn
endif

This look fine.

I.e. the 30_text_de.cf file contains this line:
lang de describe USER_IN_WELCOMELIST_TO Empfängeradresse steht in Ihrer persönlichen weißen Liste

But no if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist).

If my spamassassin does not have the feature „feature_blocklist_welcomelist“, it will fail because the rule USER_IN_WELCOMELIST_TO does not exist. Right?

My versions:
sa-update version 3.4.2 / svn1840377
  running on Perl version 5.28.1

SpamAssassin version 3.4.2
running on Perl version 5.28.1
Comment 13 harald ms 2020-07-17 17:54:38 UTC
Now "ALLOW" has become "WELCOME"
funny?

This is on Debian stretch 9.12

/etc/cron.daily/spamassassin
config: warning: description exists for non-existent rule USER_IN_WELCOMELIST_TO

channel: lint check of update failed, channel failed
sa-update failed for unknown reasons

sa-update --version
sa-update version 3.4.2 / svn1840377
  running on Perl version 5.24.1
Comment 14 harald ms 2020-07-17 18:07:03 UTC
When you, the developers make a BIG change on words like 'ALLOW' to 'WELCOME', why dont you undo the mistakes on the daily sent RULES?
Comment 15 harald ms 2020-07-17 18:32:57 UTC
""
I'll see if someone can put some eyeballs on why that isn't getting published.
""

Debian 9.12
our update script fetches the rules every day 24h.
Comment 16 Kevin A. McGrail 2020-07-17 20:46:34 UTC
Thanks all.  The fix has been committed for some time but is depending on the ruleqa/masscheck system that publishes rules.  It also is only broken on older versions of SA so it didn't get caught in QA tests.

There are no rules pending so the SA-update failing has zero concerns for any operating systems right now.
Comment 17 harald ms 2020-07-17 22:38:22 UTC
Do I have to rewrite
/etc/cron.daily/spamassassin
for this JOKE?
Comment 18 harald ms 2020-07-17 22:42:09 UTC
root@denaos:~# sa-update --nogpg --channel updates.spamassassin.org -v
Update available for channel updates.spamassassin.org: 1879297 -> 1879882
http: (curl) GET http://sa-update.spamassassin.org/1879882.tar.gz, success
http: (curl) GET http://sa-update.spamassassin.org/1879882.tar.gz.sha512, success
http: (curl) GET http://sa-update.spamassassin.org/1879882.tar.gz.sha256, success
config: warning: description exists for non-existent rule USER_IN_WELCOMELIST_TO

channel: lint check of update failed, channel failed
Update failed, exiting with code 4
Comment 19 Kevin A. McGrail 2020-07-17 22:44:49 UTC
Use the current SA 3.4.4, and you shouldn't get a failure on that warning installing sa-update. 

The warning is fairly benign and the issue is that there are queued up changes not published.  

It's not a cron issue but it is a description for non-existent rules that won't block an update in 3.4.3 and beyond.
Comment 20 Kevin A. McGrail 2020-07-18 20:51:45 UTC
I show the ruleset 1879934 was published and has the fix.  Please confirm if you can update now.
Comment 21 harald ms 2020-07-18 22:13:18 UTC
Update sa-update runs without error since 2020-07-18 morning.
Thank you!
Comment 22 Kevin A. McGrail 2020-07-20 01:45:51 UTC
svn commit -m 'More Tweaking to USER_IN_WELCOMELIST_TO' 60_whitelist.cf 50_scores.cf 
Sending        50_scores.cf
Sending        60_whitelist.cf
Transmitting file data ..
Committed revision 1880056.