Bug 7538 - Bayes_toks grows indefinitely on macOS High Sierra when using APFS
Summary: Bayes_toks grows indefinitely on macOS High Sierra when using APFS
Status: RESOLVED WONTFIX
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Learner (show other bugs)
Version: 3.4.1
Hardware: Macintosh Mac OS X
: P2 critical
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-29 14:00 UTC by hgv
Modified: 2019-11-13 17:52 UTC (History)
5 users (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description hgv 2018-01-29 14:00:19 UTC 
I have macOS High Sierra 10.13.3 using SpamAssassin 3.4.1 on Perl 5.24 installed via MacPorts. When running SA with bayes enabled, the bayes.toks files grow indefinitely until the volume is full.

These threads suggest the issue may only occur on an APFS volume which is what I'm using also: https://discussions.apple.com/thread/8203349 https://discussions.apple.com/thread/8147080

Trying to analyze the problem I noticed that the same situation occurs when running 'make test' for Mail::SpamAssassin in the bayesdbm.t test.
Comment 1 Kevin A. McGrail 2018-08-22 03:19:06 UTC 
Sorry, but I have to believe it's a bug in Apple's ServerOS
Comment 2 Sidney Markowitz 2018-09-09 01:36:47 UTC 
Adding a comment to a closed issue so the solution shows up in searches.

This is caused by Mac OS High Sierra including Berkeley DB version 1.x, which prevents CPAN DB_File from installing properly.

The workaround is to use Macports or homebrew to install a newer Berkeley DB and then install DB_File from CPAN.

See https://wiki.apache.org/spamassassin/InstallingOnMacHighSierra
Comment 3 Bill Cole 2018-09-09 04:29:33 UTC 
(In reply to Sidney Markowitz from comment #2)
> Adding a comment to a closed issue so the solution shows up in searches.
> 
> This is caused by Mac OS High Sierra including Berkeley DB version 1.x,
> which prevents CPAN DB_File from installing properly.

I am dubious of this analysis. 

1. DB_File is part of the Perl core installation (5.18) on macOS and the Perl core installation done via MacPorts (as reported.) 

2. DB_File can be updated on macOS High Sierra via CPAN without issues. 

3. SpamAssassin can be built from a fresh svn checkout (3.4 branch) without problems and bayesdbm.t passes. 

The system I have running High Sierra is NOT using APFS, a relatively new filesystem which has had a substantial number of very serious bugs since release. I suspect this is, as reported, probably APFS-dependent, and possibly fixed in an update since this was reported and since the cited discussions went quiet. 

(In reply to Kevin A. McGrail from comment #1)
> Sorry, but I have to believe it's a bug in Apple's ServerOS

There has not been a distinct "Server" version of macOS for the past 7 versions and the fact that the reporter is using a MacPorts Perl indicates that they are not using Apple's installation of Perl or SpamAssassin.
Comment 4 Sidney Markowitz 2018-09-09 05:15:49 UTC 
(In reply to Bill Cole from comment #3)

I can well believe that it only happens on APFS, but I encountered this when I tried to build SpamAssassin on my Mac yesterday for the first time since installing High Sierra and APFS. High Sierra was fully up to date. Using the system perl, I got the error when I did a make test. Switching to a perlbrew perl that had already been installed before upgrading to High Sierra also failed the make test. Trying to install a new perl failed when it tried to make DB_File, as did trying an install of DB_File from CPAN.

After installing a new Berkeley DB I was able to install DB_File from CPAN into my existing perlbrew perl, and I was able to install a new version of Perl in perlbrew. The make test of bayesdbm.t passed with both of those perls.

This is discussed as a bug in High Sierra at this link
https://discussions.apple.com/thread/8125401?answerId=32430391022
That might require a login to view, here is the Google cache version of it
https://webcache.googleusercontent.com/search?q=cache:5WaK6PFMMY8J:https://discussions.apple.com/thread/8125401

Again, all that is with APFS, with nothing contradicting that it works on HPFS disks.
Comment 5 Sidney Markowitz 2018-09-09 11:11:42 UTC 
This appears to be fixed in the system perl in the latest beta of Mac OS X Mojave (10.14)
Comment 6 Giovanni Bechis 2019-11-13 17:52:34 UTC 
For the archives:
it seems to be reproducible also with OpenBSD, it seems related to an old BerkeleyDB version + FFS2 sparse file handling.