Bug 3563 - rethink Bayes locking to avoid "db version 0" warnings
Summary: rethink Bayes locking to avoid "db version 0" warnings
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Libraries (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: Other other
: P5 normal
Target Milestone: Future
Assignee: SpamAssassin Developer Mailing List
URL: http://bugs.debian.org/cgi-bin/bugrep...
Whiteboard:
Keywords:
Depends on: 6901
Blocks:
  Show dependency tree
 
Reported: 2004-07-02 16:14 UTC by Theo Van Dinter
Modified: 2021-04-18 10:31 UTC (History)
2 users (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Theo Van Dinter 2004-07-02 16:14:59 UTC 
In one window I have an sa-learn running, learning a few thousand mails.  In
another window, I do "sa-learn --dump magic -D" and get the debug output at the
bottom of this comment.  Notice it says the bayes db version is 0, but the bayes
db is definitely version 3.  It seems as if the tie works, but then trying to
read the db fails.  If this is this is the case, then we need to essentially
lock the db for reading (or have a collective read lock anyway) too, otherwise
the data we get could be corrupted.

debug: bayes: 18360 tie-ing to DB file R/O /home/felicity/.spamassassin/bayes_toks
debug: bayes: 18360 tie-ing to DB file R/O /home/felicity/.spamassassin/bayes_seen
debug: bayes: found bayes db version 0
debug: bayes: bayes db version 0 is not able to be used, aborting!
debug: Score set 0 chosen.
debug: bayes: 18360 tie-ing to DB file R/O /home/felicity/.spamassassin/bayes_toks
debug: bayes: 18360 tie-ing to DB file R/O /home/felicity/.spamassassin/bayes_seen
debug: bayes: found bayes db version 0
debug: bayes: bayes db version 0 is not able to be used, aborting!
ERROR: Bayes dump returned an error, please re-run with -D for more information
Comment 1 spamassassin 2005-08-16 20:45:24 UTC 
Seems to be related to Debian bug #308303, which can be seen at the URL above.

It seems in 308303, there's no dumping going on, though perhaps spamd is
accessing the database.
Comment 2 Theo Van Dinter 2006-06-27 07:01:26 UTC 
After a little bit of thinking about this, I think what we'll have to do is the
multiple lock thing.  Something like:

- if a file-based DB (DBM, etc,) and
- lock_method flock

when tieing read only, lock the DB with LOCK_SH.  when tieing read/write, lock
the DB with LOCK_EX (current behavior).

Having lock_method as anything else means we'd always have to behave ala
LOCK_EX, which really just kills Bayes usage, so I'd rather deal with the
occasional odd behavior.  One issue with this plan is that I don't know if
there's a lock queue, which could potentially mean our LOCK_EX for writes may
consistently time out if reads continue to keep the shared lock in place.  It's
probably platform dependent.

Alternately, we punt this and the answer is "this behavior can happen, if you
want to avoid it use SQL".

Thoughts?
Comment 3 Justin Mason 2006-07-12 09:39:05 UTC 
what kind of db format is this?

I thought Berkeley DB files could deal with this....
Comment 4 Theo Van Dinter 2006-07-12 13:41:40 UTC 
(In reply to comment #3)
> what kind of db format is this?
> I thought Berkeley DB files could deal with this....

It's BerkeleyDB.
Comment 5 Theo Van Dinter 2006-08-13 00:34:57 UTC 
if we ever get movement on this, let's backport to 3.1
Comment 6 Justin Mason 2007-02-21 12:05:59 UTC 
pushing out to 3.3.0, since I don't think it's a 3.2.0 blocker. shout (or change
the milestone) if you disagree....
Comment 7 Justin Mason 2010-01-27 02:21:16 UTC 
moving most remaining 3.3.0 bugs to 3.3.1 milestone
Comment 8 Justin Mason 2010-01-27 03:16:53 UTC 
reassigning, too
Comment 9 Justin Mason 2010-03-23 16:34:11 UTC 
moving all open 3.3.1 bugs to 3.3.2
Comment 10 Karsten Bräckelmann 2010-03-23 17:43:10 UTC 
Moving back off of Security, which got changed by accident during the mass Target Milestone move.
Comment 11 Rob Mueller 2013-02-18 21:37:43 UTC 
I think this error is due to something else entirely. I've created a separate bug here: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6901
Comment 12 Henrik Krohns 2021-04-18 10:31:51 UTC 
Closing as Bug 6901 supposedly fixed this.