Bug 8092 - compat plan for whitelist/welcomelist is confusing
Summary: compat plan for whitelist/welcomelist is confusing
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Documentation (show other bugs)
Version: 4.0.0
Hardware: PC NetBSD
: P2 minor
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-12-15 12:52 UTC by Greg Troxel
Modified: 2023-08-04 16:39 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Greg Troxel 2022-12-15 12:52:35 UTC
The change from white to welcome is confusing and not explained enough in the release notes and wiki page.

Specifically, there seems to be compat for "whitelist_from" etc. in config files, and that seems to be entirely separate from what rules are called.  This is sort of explained, but not really.   The missing point for users is that the rule renaming in enable_compat has nothing to do with config file contents.  Or maybe I'm wrong.

The new init.pre has enable_compat welcomelist_blocklist.  However init.pre is said to be where customizations go, so people are going to end up without that in their config, as did I.  (There's a larger issue that config files should be either from the distribution not to be modified, or wholly for users to modify, to play nice with packaging.)   The wiki text does not explain that to get the new behavior you have to add a compat option.  This needs to be hit-over-the-head clear because it does not make sense to have to turn on a compat option to get the new standard behavior.

It seems obvious to me, but maybe I'm wrong, that the enable_compat option is backwards.  If not set, one should just get the new names.  If something is set, then the old names should appear as well.

It is unclear to people why they should choose or not choose to enable_compat.  As I see it, it should be enabled (new rules only), for most people, and the reasons not to are if you have config that rescores those rules, or meta rules that reference them, or you are parsing SA output and looking for them.   All of that should be fixed, and compat to be able to fix async is of course fine.

For the release, what's needed is a better explanation in release notes and wiki, so that a clone of me starting over from going to read would not be confused.

I would also want to flip the compat option to be whitelist_blacklist, invert the sense, and not put it in init.pre.   Thus people that want the old names have to turn it on, and people that don't need them (99%?) don't need to do anything.

(Rewritten from a users@ post after KAM asked to file a bug.)
Comment 1 Henrik Krohns 2022-12-17 09:21:03 UTC
To clarify few things:

(In reply to Greg Troxel from comment #0)
> The change from white to welcome is confusing and not explained enough in
> the release notes and wiki page.
> 
> Specifically, there seems to be compat for "whitelist_from" etc. in config
> files, and that seems to be entirely separate from what rules are called. 
> This is sort of explained, but not really.   The missing point for users is
> that the rule renaming in enable_compat has nothing to do with config file
> contents.  Or maybe I'm wrong.

As described in enable_compat documentation (man Mail::SpamAssassin::Conf), the only thing "enable_compat welcomelist_blocklist" does is enable using "if !can(Mail::SpamAssassin::Conf::compat_welcomelist_blocklist)" style checks in rules.

When it's used, all *WHITELIST/*BLACKLIST rules from stock ruleset are ignored by the if blocks and only new rule names are used.

> The new init.pre has enable_compat welcomelist_blocklist.  However init.pre
> is said to be where customizations go, so people are going to end up without
> that in their config, as did I.  (There's a larger issue that config files
> should be either from the distribution not to be modified, or wholly for
> users to modify, to play nice with packaging.)   The wiki text does not
> explain that to get the new behavior you have to add a compat option.  This
> needs to be hit-over-the-head clear because it does not make sense to have
> to turn on a compat option to get the new standard behavior.

The reason for suggesting to put it in init.pre, is because that's where it's put by default in 4.0. You _can_ put it in any *.pre file, it makes no difference. Maybe we can just suggest putting it in local.pre do it's not so confusing.

Distribution or manual SA installs/upgrades never overwrite existing *.pre files, this is by design.

> It seems obvious to me, but maybe I'm wrong, that the enable_compat option
> is backwards.  If not set, one should just get the new names.  If something
> is set, then the old names should appear as well.

We can't just suddently remove USER_IN_WHITELIST etc for all users, they might be using the old names in metas etc.

For a fresh install, it comes enabled in init.pre by default. In this case it's probably safe to assume that the user does not have any local rules that refer to the old names.

> It is unclear to people why they should choose or not choose to
> enable_compat.  As I see it, it should be enabled (new rules only), for most
> people, and the reasons not to are if you have config that rescores those
> rules, or meta rules that reference them, or you are parsing SA output and
> looking for them.   All of that should be fixed, and compat to be able to
> fix async is of course fine.
> 
> For the release, what's needed is a better explanation in release notes and
> wiki, so that a clone of me starting over from going to read would not be
> confused.
> 
> I would also want to flip the compat option to be whitelist_blacklist,
> invert the sense, and not put it in init.pre.   Thus people that want the
> old names have to turn it on, and people that don't need them (99%?) don't
> need to do anything.

It was named enable_compat to enable declare compatibility for a feature. Old SA versions are not compatible with welcomelist/blocklist naming (eval-functions etc). You are required to set the option yourself, to declare that your local .cf configuration is compatible with welcomelist/blocklist and does not refer to the old names and functions.

I'll see if I can make it more clear on the wiki.
Comment 2 Henrik Krohns 2022-12-17 09:56:33 UTC
Anything missing?

https://cwiki.apache.org/confluence/display/SPAMASSASSIN/WelcomelistBlocklist
Comment 3 Henrik Krohns 2022-12-17 10:24:10 UTC
I've added a hopefully full list of all changed names to the Wiki for reference.
Comment 4 Greg Troxel 2022-12-17 14:48:40 UTC
The part I did not understand is that enable_compat is a declaration by the user that their rules are compatible with the new feature, and that the new feature should be partially off by default.  I expected that without compat set, one would simply get the new names, only, and that you could turn on compatibility to function with the old names.  In the future, this would fade and not need to be carried indefinitely, but I realize that can happen with rule changes.  SO it would be good to say more loudly where the sense of the compat comes from, as I expect it will feel backwards to most people, but maybe that's just me.

The new page is vastly improved - thanks!  The remaining point that is not explained is how whitelist_from is handled.  I know from discussions and reading that it is treated as a synonym for welcomelist_from, silently, and that this might change in 4.1.  Right now the new page mostly talks about rules (which is the hard part, I realize), and doesn't give guidance about changing directives in local config.
Comment 5 Henrik Krohns 2022-12-20 07:02:22 UTC
This can probably be closed, additional comments can of course be posted.
Comment 6 John Brooks 2023-08-04 16:39:11 UTC
Please just ensure the old names continue working. I don't want to have to rejig my config files for this on the next release.