Created attachment 4085 [details]
Procmail logfile showing errors

Created attachment 4086 [details]
spamassassin -D --lint output from a message that produced errors via procmail

Created attachment 4087 [details]
Spam message used in the test run.

hey mrneutron!

this looks a lot like an NFS problem, namely intermittent failure to load files:

Can't locate Mail/SpamAssassin/Util/Progress.pm in @INC (@INC contains:
/home/mailfilter/local/share/perl/5.8.4 /etc/perl /usr/local/lib/perl/5.8.4
/usr/local/share/perl/5.8.4 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8
/usr/share/perl/5.8 /usr/local/lib/site_perl) at
/mnt/slammer/vol/boot/spunky/hopei/mailfilter/local/bin/spamassassin line 81.
BEGIN failed--compilation aborted at
/mnt/slammer/vol/boot/spunky/hopei/mailfilter/local/bin/spamassassin line 81.
procmail: Program failure (2) of
"/mnt/slammer/vol/boot/spunky/hopei/mailfilter/local/bin/spamassassin"

check: no loaded plugin implements 'check_main': cannot scan! at
/mnt/slammer/vol/boot/spunky/hopei/mailfilter/local/share/perl/5.8.4/Mail/SpamAssassin/PerMsgStatus.pm
line 164.
procmail: Program failure (2) of
"/mnt/slammer/vol/boot/spunky/hopei/mailfilter/local/bin/spamassassin"

is that possible?

If you could capture -D output from one of these failing cases, that'd probably
help, too.  (-D output goes to stderr, whereas filtered text goes to stdout, so
it's possible to log -D output to a separate file from the procmailrc.)

> is that possible?

It's dreamhost.  Their servers and filers are oversold and constantly running at
more than full capacity so it's entirely possible.

> If you could capture -D output from one of these failing cases, that'd probably
> help, too.  (-D output goes to stderr, whereas filtered text goes to stdout, so
> it's possible to log -D output to a separate file from the procmailrc.)

I'd love to do this but I don't know how.  Can you give me a pointer?

(In reply to comment #5)
> > is that possible?
> 
> It's dreamhost.  Their servers and filers are oversold and constantly running at
> more than full capacity so it's entirely possible.

ah, yep.  on my DH account, references to /vol/boot/ are all NFS volumes, so
I'll bet it's NFS problems.
 
> > If you could capture -D output from one of these failing cases, that'd probably
> > help, too.  (-D output goes to stderr, whereas filtered text goes to stdout, so
> > it's possible to log -D output to a separate file from the procmailrc.)
> 
> I'd love to do this but I don't know how.  Can you give me a pointer?

sure -- lemme see if I can get some time together to do that RSN.

ok, replace the "| spamassassin" stanza in your .procmailrc with something like
this:

:0fw: spamassassin.lock
* < 512000
| spamassassin -D 2>>spamassassin.log


that should write a debug logfile to "spamassassin.log".  the thing to look out
for, then, is cases of files being there in some scans (in the "dbg: config:
read file" part of the log), and not appearing in others.

Here's another idea; would it be possible to copy the spamassassin lib dirs
(${HOME}/local/share/spamassassin, etc.) to /tmp ?   /tmp is probably always a
local disk, therefore immune to NFS nastiness.  (may be best to confirm that
it's caused by NFS first though.)

FYI, the server in your example (spunkymail-mx9.g.dreamhost.com) was having
issues with homefs at least since last Thursday (what actually maps your home
directory to /home/<user> on their system).  I submitted a ticket earlier this
week and they said that they had it resolved.  So, if you were using your own
version of spamassassin, installed in /home/<user>, then it's likely that you
wouldn't be able to find all of the required perl libs.  I tried using whatever
HOME var was passed on the mail server (/mnt/polisher/vol/boot/...) so that even
if homefs wasn't working, things would still work, but wasn't able to
successfully change where spamassassin looks for libs.  

If it still isn't working for you, I'd submit a ticket and explain what mail
server the message came from and say that your home directory is most likely not
being properly mapped with homefs.  They should fix it up.

If you do find a good long-term solution, please let me know about it.

-Brian

I'm having the same problems, only I've tried version 3.2.5 that causes the break!

Jun 30 12:27:23 beta spamd[30598]: logger: removing stderr method
Jun 30 12:27:24 beta spamd[30600]: check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164.

Any ideas?

James

Sorry... more information listed below:
(1)  Using spamass-milter 0.3.1
(2)  Setup works if I back down to 3.2.4 of spamassassin
(3)  'make test' reports no errors with spamassassin 3.2.5...

Is there something that has changed in the interface with spamass-milter that could be casing this???

Actually, I think it may be more of a problem with the perl end of spamassassin.  Though again I'm not sure.  Spamassassin just refuses to run as a deamon is the symptom.

James

Here is a log with 3.2.5 ... with 'spamd -D -d'


-- log
Jun 30 15:34:48 beta spamd[9222]: logger: successfully added syslog method
Jun 30 15:34:48 beta spamd[9222]: spamd: will perform setuids? 1
Jun 30 15:34:48 beta spamd[9222]: spamd: creating INET socket:
Jun 30 15:34:49 beta spamd[9222]: spamd: Listen: 128
Jun 30 15:34:49 beta spamd[9222]: spamd: LocalAddr: 127.0.0.1
Jun 30 15:34:49 beta spamd[9222]: spamd: LocalPort: 783
Jun 30 15:34:49 beta spamd[9222]: spamd: Proto: 6
Jun 30 15:34:50 beta spamd[9222]: spamd: ReuseAddr: 1
Jun 30 15:34:50 beta spamd[9222]: spamd: Type: 1
Jun 30 15:34:50 beta spamd[9222]: logger: adding facilities: all
Jun 30 15:34:50 beta spamd[9222]: logger: logging level is DBG
Jun 30 15:34:51 beta spamd[9222]: generic: SpamAssassin version 3.2.5
Jun 30 15:34:51 beta spamd[9222]: config: score set 0 chosen.
Jun 30 15:34:51 beta spamd[9222]: dns: no ipv6
Jun 30 15:34:51 beta spamd[9222]: dns: is Net::DNS::Resolver available? yes
Jun 30 15:34:52 beta spamd[9222]: dns: Net::DNS version: 0.62
Jun 30 15:34:52 beta spamd[9222]: logger: removing stderr method
Jun 30 15:34:52 beta spamd[9224]: spamd: successfully daemonized
Jun 30 15:34:52 beta spamd[9224]: spamd: Preloading modules with HOME=/tmp/spamd-9224-init
Jun 30 15:34:53 beta spamd[9224]: ignore: test message to precompile patterns and load modules
Jun 30 15:34:53 beta spamd[9224]: config: using "/etc/mail/spamassassin" for site rules pre files
Jun 30 15:34:53 beta spamd[9224]: config: using "/usr/share/spamassassin" for sys rules pre files
Jun 30 15:34:53 beta spamd[9224]: config: using "/usr/share/spamassassin" for default rules dir
Jun 30 15:34:54 beta spamd[9224]: config: read file /usr/share/spamassassin/72_removed.cf
Jun 30 15:34:54 beta spamd[9224]: config: using "/etc/mail/spamassassin" for site rules dir
Jun 30 15:34:54 beta spamd[9224]: config: read file /etc/mail/spamassassin/local.cf
Jun 30 15:34:54 beta spamd[9224]: rules: AXB_RCVD_ZOOBSEND merged duplicates: BROKEN_RATWARE_BOM DEAR_HOMEOWNER DIV_CENTER_A_HREF DRUG_RA_PRICE FM_DDDD_TIMES_2 FM_SEX_HOSTDDDD HS_PHARMA_1 HS_UPLOADED_SOFTWARE OEBOUND STOX_RCVD_N_NN_N URIBL_RHS_ABUSE URIBL_RHS_BOGUSMX URIBL_RHS_DSN URIBL_RHS_POST URIBL_RHS_TLD_WHOIS URIBL_RHS_WHOIS URIBL_XS_SURBL URI_L_PHP XMAILER_MIMEOLE_OL_5E7ED XMAILER_MIMEOLE_OL_C7C33 XMAILER_MIMEOLE_OL_D03AB X_LIBRARY YOUR_CRD_RATING
Jun 30 15:34:55 beta spamd[9224]: conf: finish parsing
Jun 30 15:34:55 beta spamd[9224]: bayes: no dbs present, cannot tie DB R/O: /tmp/spamd-9224-init/.spamassassin/bayes_toks
Jun 30 15:34:55 beta spamd[9224]: config: score set 1 chosen.
Jun 30 15:34:56 beta spamd[9224]: message: main message type: text/plain
Jun 30 15:34:56 beta spamd[9224]: message: ---- MIME PARSER START ----
Jun 30 15:34:56 beta spamd[9224]: message: parsing normal part
Jun 30 15:34:56 beta spamd[9224]: message: ---- MIME PARSER END ----
Jun 30 15:34:57 beta spamd[9224]: bayes: no dbs present, cannot tie DB R/O: /tmp/spamd-9224-init/.spamassassin/bayes_toks
Jun 30 15:34:57 beta spamd[9224]: check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164.

This seems like a busted install.

What does spamassassin -D --lint show?

Also, do you possibly have two different copies of spamd on your system?

What are the contents of your .pre files in /etc/mail/spamassassin?

--- Output
[root@beta redhat]# spamassassin -D --lint
[19623] dbg: logger: adding facilities: all
[19623] dbg: logger: logging level is DBG
[19623] dbg: generic: SpamAssassin version 3.2.5
[19623] dbg: config: score set 0 chosen.
[19623] dbg: util: running in taint mode? yes
[19623] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH
[19623] dbg: util: PATH included '/usr/kerberos/sbin', keeping
[19623] dbg: util: PATH included '/usr/kerberos/bin', keeping
[19623] dbg: util: PATH included '/usr/local/sbin', keeping
[19623] dbg: util: PATH included '/usr/local/bin', keeping
[19623] dbg: util: PATH included '/sbin', keeping
[19623] dbg: util: PATH included '/bin', keeping
[19623] dbg: util: PATH included '/usr/sbin', keeping
[19623] dbg: util: PATH included '/usr/bin', keeping
[19623] dbg: util: PATH included '/usr/X11R6/bin', keeping
[19623] dbg: util: PATH included '/root/bin', keeping
[19623] dbg: util: final PATH set to: /usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin
[19623] dbg: dns: no ipv6
[19623] dbg: dns: is Net::DNS::Resolver available? yes
[19623] dbg: dns: Net::DNS version: 0.62
[19623] dbg: diag: perl platform: 5.010000 linux
[19623] dbg: diag: module installed: Digest::SHA1, version 2.11
[19623] dbg: diag: module installed: HTML::Parser, version 3.56
[19623] dbg: diag: module installed: Net::DNS, version 0.62
[19623] dbg: diag: module installed: MIME::Base64, version 3.07_01
[19623] dbg: diag: module installed: DB_File, version 1.817
[19623] dbg: diag: module installed: Net::SMTP, version 2.31
[19623] dbg: diag: module not installed: Mail::SPF ('require' failed)
[19623] dbg: diag: module installed: Mail::SPF::Query, version 1.999001
[19623] dbg: diag: module not installed: IP::Country::Fast ('require' failed)
[19623] dbg: diag: module not installed: Razor2::Client::Agent ('require' failed)
[19623] dbg: diag: module not installed: Net::Ident ('require' failed)
[19623] dbg: diag: module not installed: IO::Socket::INET6 ('require' failed)
[19623] dbg: diag: module not installed: IO::Socket::SSL ('require' failed)
[19623] dbg: diag: module installed: Compress::Zlib, version 2.011
[19623] dbg: diag: module installed: Time::HiRes, version 1.9715
[19623] dbg: diag: module not installed: Mail::DomainKeys ('require' failed)
[19623] dbg: diag: module not installed: Mail::DKIM ('require' failed)
[19623] dbg: diag: module installed: DBI, version 1.605
[19623] dbg: diag: module installed: Getopt::Long, version 2.37
[19623] dbg: diag: module installed: LWP::UserAgent, version 2.036
[19623] dbg: diag: module installed: HTTP::Date, version 1.47
[19623] dbg: diag: module installed: Archive::Tar, version 1.38
[19623] dbg: diag: module installed: IO::Zlib, version 1.09
[19623] dbg: diag: module not installed: Encode::Detect ('require' failed)
[19623] dbg: ignore: using a test message to lint rules
[19623] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
[19623] dbg: config: using "/usr/share/spamassassin" for sys rules pre files
[19623] dbg: config: using "/usr/share/spamassassin" for default rules dir
[19623] dbg: config: read file /usr/share/spamassassin/72_removed.cf
[19623] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[19623] dbg: config: read file /etc/mail/spamassassin/local.cf
[19623] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
[19623] dbg: config: read file /root/.spamassassin/user_prefs
[19623] dbg: rules: AXB_RCVD_ZOOBSEND merged duplicates: BROKEN_RATWARE_BOM DEAR_HOMEOWNER DIV_CENTER_A_HREF DRUG_RA_PRICE FM_DDDD_TIMES_2 FM_SEX_HOSTDDDD HS_PHARMA_1 HS_UPLOADED_SOFTWARE OEBOUND STOX_RCVD_N_NN_N URIBL_RHS_ABUSE URIBL_RHS_BOGUSMX URIBL_RHS_DSN URIBL_RHS_POST URIBL_RHS_TLD_WHOIS URIBL_RHS_WHOIS URIBL_XS_SURBL URI_L_PHP XMAILER_MIMEOLE_OL_5E7ED XMAILER_MIMEOLE_OL_C7C33 XMAILER_MIMEOLE_OL_D03AB X_LIBRARY YOUR_CRD_RATING
[19623] dbg: conf: finish parsing
[19623] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks
[19623] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen
[19623] dbg: bayes: found bayes db version 3
[19623] dbg: bayes: DB journal sync: last sync: 0
[19623] dbg: bayes: not available for scanning, only 0 spam(s) in bayes DB < 200
[19623] dbg: bayes: untie-ing
[19623] dbg: config: score set 0 chosen.
[19623] dbg: message: main message type: text/plain
[19623] dbg: message: ---- MIME PARSER START ----
[19623] dbg: message: parsing normal part
[19623] dbg: message: ---- MIME PARSER END ----
[19623] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks
[19623] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen
[19623] dbg: bayes: found bayes db version 3
[19623] dbg: bayes: DB journal sync: last sync: 0
[19623] dbg: bayes: not available for scanning, only 0 spam(s) in bayes DB < 200
[19623] dbg: bayes: untie-ing
check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164.

--  ONLY ONE INSTALLED
[root@beta redhat]# whereis spamassassin
spamassassin: /usr/bin/spamassassin /usr/share/spamassassin /usr/share/man/man1/spamassassin.1.gz
[root@beta redhat]# ls /usr/share/spamassassin
72_removed.cf
[root@beta redhat]# ls /usr/bin/spamassassin
/usr/bin/spamassassin

-- NO PRE files... hmmm. will have to check this
[root@beta redhat]# ls /etc/mail/spamassassin/
local.cf  spamassassin-default.rc  spamassassin-helper.sh  spamassassin-spamc.rc

Thanks!
My problem was the missing init.pre file in /etc/mail/spamassassin.

James

Closing old bugs. Server and user problems.