Bug 7962 - Deprecate sa-compile in 4.0.0
Summary: Deprecate sa-compile in 4.0.0
Status: RESOLVED WONTFIX
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: sa-compile (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: All All
: P2 normal
Target Milestone: 4.0.0
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-06 14:11 UTC by Henrik Krohns
Modified: 2022-03-09 14:35 UTC (History)
2 users (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Henrik Krohns 2022-03-06 14:11:27 UTC 
I'm calling vote for officially deprecating sa-compile in 4.0.0.

While it might have been a novel concept over decade ago, time has passed it and any "savings" it might give. We have no manpower to support all the complex inner workings of BodyRuleBaseExtractor/re2c/etc.

We can keep the code in trunk for now, but should clearly mark everything as deprecated without further support. Tempted to remove anything related to it in 4.0 svn branch (if we create one).

+1
Comment 1 Bill Cole 2022-03-06 18:02:50 UTC 
(In reply to Henrik Krohns from comment #0)
> I'm calling vote for officially deprecating sa-compile in 4.0.0.
> 
> While it might have been a novel concept over decade ago, time has passed it
> and any "savings" it might give.

I don't doubt this, but I expect we might get some resistance from people who have used it as part of their routine workflow for so long, based on that obsolete premise. Do you know of any specific test which will confirm that it is no longer worthwhile?
Comment 2 Henrik Krohns 2022-03-06 18:43:55 UTC 
(In reply to Bill Cole from comment #1)
> I don't doubt this, but I expect we might get some resistance from people
> who have used it as part of their routine workflow for so long, based on
> that obsolete premise. Do you know of any specific test which will confirm
> that it is no longer worthwhile?

It can provide marginal speed and memory savings, at the cost of lots of complex codebase and tools that can break. Makes no difference to me if someone would resist or not, it's up to the project to decide what it can and wants to support. A bit of the same case as TxRep, a thing just barely works.. and as such I would maybe keep it distributed, but with a "use at your own risk" clause.
Comment 3 Henrik Krohns 2022-03-09 10:22:55 UTC 
Some quick benchmarks with 100 emails

sa-compile stock
18.47user 0.15system 0:18.62elapsed 99%CPU (0avgtext+0avgdata 129804maxresident)k
18.49user 0.11system 0:18.61elapsed 99%CPU (0avgtext+0avgdata 128388maxresident)k

stock
20.95user 0.08system 0:21.04elapsed 99%CPU (0avgtext+0avgdata 126452maxresident)k
20.75user 0.21system 0:20.97elapsed 99%CPU (0avgtext+0avgdata 124616maxresident)k

sa-compile massive rules
37.11user 0.32system 0:37.91elapsed 98%CPU (0avgtext+0avgdata 231652maxresident)k
37.48user 0.23system 0:37.82elapsed 99%CPU (0avgtext+0avgdata 232732maxresident)k

massive rules
54.89user 0.18system 0:55.19elapsed 99%CPU (0avgtext+0avgdata 211108maxresident)k
53.17user 0.28system 0:53.57elapsed 99%CPU (0avgtext+0avgdata 210512maxresident)k

Maybe 10% speedup with completely stock trunk (<400 body rules). 30% when there are heaps of extra rules (heinlein, KAM etc, 3000+ body rules).

I guess I'll spend a little more of my holiday verifying that mass-check results are identical with and without sa-compile, with regards to all the UTF-8 stuff etc..
Comment 4 Henrik Krohns 2022-03-09 14:35:09 UTC 
Forget about it, I guess most issues are now fixed.

Sending        spamassassin-3.4/lib/Mail/SpamAssassin/Plugin/BodyRuleBaseExtractor.pm
Sending        spamassassin-3.4/sa-compile.raw
Sending        trunk/sa-compile.raw
Sending        trunk/t/sa_compile.t
Transmitting file data ....done
Committing transaction...
Committed revision 1898791.