Bug 3838 - [review] allow_user_rules causes 'Insecure dependency in eval while running setuid' - caused by Perl taint bug
[review] allow_user_rules causes 'Insecure dependency in eval while running s...
Status: RESOLVED FIXED
Product: Spamassassin
Classification: Unclassified
Component: spamassassin
3.0.3
PC Linux
: P5 minor
: 3.1.2
Assigned To: Daryl C. W. O'Shea
ready for commit
:
: 3966 4445 (view as bug list)
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2004-09-28 10:30 UTC by Tony
Modified: 2006-05-22 21:24 UTC (History)
7 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
Email that passed SA due to a SA error. message/rfc822 None David Gibbs [NoCLA]
spamassassin -D --lint output text/plain None David Gibbs [NoCLA]
My user_prefs file text/plain None Jim Smith [NoCLA]
patch patch None Daryl C. W. O'Shea [HasCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Tony 2004-09-28 10:30:52 UTC
Running Linux Redhat 9 Kernel 2.4.20-31.9. Installed Spamassassin 3.0 getting 
this debug error message. Spam emails are getting in when this error occurs.
CSPAN said my PerMsgStatus.pm is up to date. Thanks for your help.

Tony.

Sep 28 07:03:55 metracom spamd[27473]: logmsg: error: Insecure dependency in 
eval while running setuid /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMs
gStatus.pm line 1669, <GEN76> line 55._ , continuing
Comment 1 Theo Van Dinter 2004-09-28 10:48:08 UTC
Subject: Re:  New: Insecure dependency in eval while running setuid

On Tue, Sep 28, 2004 at 10:30:53AM -0700, bugzilla-daemon@bugzilla.spamassassin.org wrote:
> Running Linux Redhat 9 Kernel 2.4.20-31.9. Installed Spamassassin 3.0 getting 
> this debug error message. Spam emails are getting in when this error occurs.
> CSPAN said my PerMsgStatus.pm is up to date. Thanks for your help.
> 
> Sep 28 07:03:55 metracom spamd[27473]: logmsg: error: Insecure dependency in 
> eval while running setuid /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMs
> gStatus.pm line 1669, <GEN76> line 55._ , continuing

That's the eval running the body rules.  There shouldn't be anything tainted
while in there, or else both spamassassin and spamd would fail constantly.

Is this a vanilla 3.0.0 install, or do you have any rules/patches/etc applied?
Was it a clean install or an upgrade?  Do you get the error all the time, or
only on certain messages?  All users or just one/a few?

Need more information. :)

Comment 2 David Gibbs 2004-10-25 19:19:22 UTC
I'm encountering the same problem ... it only seems to manifest when I set
"skip_rbl_checks" to 0.
Comment 3 Bob Menschel 2005-04-09 14:38:32 UTC
If Tony, David, or anyone else could attach an email that can be used to
reproduce this problem to this bug, and your config files, we might be able to
track down the cause...
Comment 4 David Gibbs 2005-05-07 10:27:55 UTC
Created attachment 2854 [details]
Email that passed SA due to a SA error.

It started happening again for me ... with SpamAssassin version 3.0.3 running
on Perl version 5.8.0.

Exact log message is: May  6 17:50:38 linux spamd[3396]: error: Insecure
dependency in eval while running setuid at
/usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line
1958, <GEN349> line 57._ Transport endpoint is not connected, continuing

The problem is intermittent ... I cannot reproduce on demand.

I've attached a sample email ... but when I feed this email through SA
manually, it works fine and flags it as spam (rather high scoring).

What does the error message mean?
Comment 5 David Gibbs 2005-05-07 10:56:05 UTC
I was able to run spamd in debug mode for a while and had a spam message slip by
... here's the log output...

May  7 12:47:32 linux sendmail[18895]: j47HlDVu018895:
from=<toby.reid_dv@main.unibanka.lv>, size=875, class=0, nrcpts=1,
msgid=<FHOPIAHOBFHEMBLJHEMLMPOPPHAA.toby.reid_dv@main.unibanka.lv>, proto=ESMTP,
daemon=external-msa28-25, relay=p54A25C29.dip.t-dialin.net [84.162.92.41]
May  7 12:47:32 linux clamav-milter[18896]: j47HlDVu018895: clean message from
<toby.reid_dv@main.unibanka.lv>
May  7 12:47:32 linux sendmail[18895]: j47HlDVu018895: Milter add: header:
X-Virus-Scanned: ClamAV version 0.84, clamav-milter version 0.84e on
linux.midrange.com
May  7 12:47:32 linux sendmail[18895]: j47HlDVu018895: Milter add: header:
X-Virus-Status: Clean
May  7 12:47:32 linux sendmail[18895]: j47HlDVu018895: Milter add: header:
X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-1.7.2
(mail.midrange.com [69.3.23.28]); Sat, 07 May 2005 12:47:32 -0500 (CDT)
May  7 12:47:32 linux spamd[18086]: logmsg: connection from localhost
[127.0.0.1] at port 52118 
May  7 12:47:32 linux spamd[18086]: connection from localhost [127.0.0.1] at
port 52118 
May  7 12:47:32 linux spamd[18086]: logmsg: info: setuid to david succeeded 
May  7 12:47:32 linux spamd[18086]: info: setuid to david succeeded 
May  7 12:47:32 linux spamd[18086]: debug: finishing parsing! 
May  7 12:47:32 linux spamd[18086]: debug: user has changed 
May  7 12:47:32 linux spamd[18086]: debug: bayes: Using username: david 
May  7 12:47:32 linux spamd[18086]: debug: bayes: Database connection established 
May  7 12:47:32 linux spamd[18086]: debug: bayes: found bayes db version 3 
May  7 12:47:32 linux spamd[18086]: debug: bayes: Using userid: 4 
May  7 12:47:32 linux spamd[18086]: debug: Score set 3 chosen. 
May  7 12:47:32 linux spamd[18086]: logmsg: processing message
<FHOPIAHOBFHEMBLJHEMLMPOPPHAA.toby.reid_dv@main.unibanka.lv> for david:501. 
May  7 12:47:32 linux spamd[18086]: processing message
<FHOPIAHOBFHEMBLJHEMLMPOPPHAA.toby.reid_dv@main.unibanka.lv> for david:501. 
May  7 12:47:32 linux spamd[18086]: debug: received-header: parsed as [
ip=84.162.92.41 rdns=p54A25C29.dip.t-dialin.net helo=one.se by=mail.midrange.com
ident= envfrom= intl=0 id=j47HlDVu018895 auth= ] 
May  7 12:47:32 linux spamd[18086]: debug: received-header: relay 84.162.92.41
trusted? no internal? no 
May  7 12:47:32 linux spamd[18086]: debug: metadata: X-Spam-Relays-Trusted:  
May  7 12:47:32 linux spamd[18086]: debug: metadata: X-Spam-Relays-Untrusted: [
ip=84.162.92.41 rdns=p54A25C29.dip.t-dialin.net helo=one.se by=mail.midrange.com
ident= envfrom= intl=0 id=j47HlDVu018895 auth= ] 
May  7 12:47:32 linux spamd[18086]: debug: ---- MIME PARSER START ---- 
May  7 12:47:32 linux spamd[18086]: debug: main message type: text/plain 
May  7 12:47:32 linux spamd[18086]: debug: parsing normal part 
May  7 12:47:32 linux spamd[18086]: debug: added part, type: text/plain 
May  7 12:47:32 linux spamd[18086]: debug: ---- MIME PARSER END ---- 
May  7 12:47:32 linux spamd[18086]: debug: decoding: other encoding type (8bit),
ignoring 
May  7 12:47:32 linux spamd[18086]: debug: Language possibly: sco,en,fy,af 
May  7 12:47:32 linux spamd[18086]: debug: metadata: X-Languages: sco en fy af 
May  7 12:47:32 linux spamd[18086]: debug: metadata: X-Relay-Countries: XX 
May  7 12:47:32 linux spamd[18086]: debug: uri found:
http://exorcized.net/rm.php?special 
May  7 12:47:32 linux spamd[18086]: debug: uri found:
http://exorcized.net/cs/?special 
May  7 12:47:32 linux spamd[18086]: debug: Running tests for priority: 0 
May  7 12:47:32 linux spamd[18086]: debug: running header regexp tests; score so
far=0 
May  7 12:47:32 linux spamd[18086]: debug: all '*From' addrs:
toby.reid_dv@main.unibanka.lv 
May  7 12:47:32 linux spamd[18086]: debug: all '*To' addrs: manager@midrange.com 
May  7 12:47:32 linux spamd[18086]: debug: forged-HELO: from=t-dialin.net
helo=one.se by=midrange.com 
May  7 12:47:32 linux spamd[18086]: debug: forged-HELO: mismatch on HELO:
'one.se' != 't-dialin.net' 
May  7 12:47:32 linux spamd[18086]: debug: running body-text per-line regexp
tests; score so far=0 
May  7 12:47:32 linux spamd[18086]: debug: running uri tests; score so far=0 
May  7 12:47:32 linux spamd[18086]: logmsg: error: Insecure dependency in eval
while running setuid at
/usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 1958,
<GEN13> line 44._ , continuing 
May  7 12:47:32 linux spamd[18086]: error: Insecure dependency in eval while
running setuid at
/usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 1958,
<GEN13> line 44._ , continuing 
May  7 12:47:32 linux sendmail[18919]: j47HlDVu018895: to=david, delay=00:00:01,
xdelay=00:00:00, mailer=local, pri=61334, dsn=2.0.0, stat=Sent
May  7 12:47:32 linux spamd[18083]: logmsg: connection from localhost
[127.0.0.1] at port 52120
Comment 6 Bob Menschel 2005-07-16 11:20:10 UTC
Triage: Not reproducible on demand, but apparently does happen enough to
occasionally get a debug on it. Unlikely it can be fixed for 3.1.0, and happens
rarely enough that it's not a critical problem. Setting target milestone to
3.1.1. Note that bug 3966 seems to report the same problem (closing that one as
dupe), and bug 4445 looks very similar and may be related. 
Comment 7 Bob Menschel 2005-07-16 11:20:22 UTC
*** Bug 3966 has been marked as a duplicate of this bug. ***
Comment 8 Loren Wilton 2005-07-16 21:26:19 UTC
Subject: Re:  Insecure dependency in eval while running setuid

> happens
> rarely enough that it's not a critical problem.

Bloody annoying though.  I have it happen 5-20 times a day on about 200
spams a day.

It will happen on any of the evals for user rules: body, rawbody, full, and
I suspect header too.  So far I haven't been able to figure out enough Perl
so that I can get any extra debugging out and find out what it is really
complaining about.

I suspect that if someone wanted to reproduce it, all they would need to do
is turn on user rules and drop any of the standard rules files (to get a
bunch of rules) into their user_prefs file.

Comment 9 David Gibbs 2005-08-23 17:53:55 UTC
(In reply to comment #8)
> Bloody annoying though.  I have it happen 5-20 times a day on about 200
> spams a day.

Seems to happen in batches too.  My system will go days without it happening and
then all of a sudden I get a batch of 15-20 spams that get through without any
SA markups.
Comment 10 David Gibbs 2005-08-25 17:48:33 UTC
(In reply to comment #6)
> Triage: Not reproducible on demand, but apparently does happen enough to
> occasionally get a debug on it. Unlikely it can be fixed for 3.1.0, and happens
> rarely enough that it's not a critical problem. 

It's been happening a lot more recently ... perhaps due to a general increase in
spam, but it's getting to an annoyance point again.
Comment 11 Mark Martinec 2005-09-05 10:17:15 UTC
Please get rid of perl 5.8.0 first, and try with 5.8.2 or later. 
 
There are lots of problems with 5.8.0, including buggy taint processing. 
Later versions have some of these problems fixed, but not all, so an 
application running in taint mode needs to provide some workarounds 
for these. It is not worth the effort trying to fix an application 
to work on Perl 5.8.0 if it works properly with later versions. 
 
One of the most annoying taint bugs in Perl (still present even in 
versions beyond 5.8.2) is that during certain regexp evaluations, 
global variables like $1, $2, $3, ... become tainted out of thin air. 
This has far-fetched consequences, as from such a point on, a classical 
untaint operation like: $str=$1 if $str=~/^(.*)\z/ no longer works 
as untaint, because $1 somehow got stuck with a taint flag. 
 
I have been fighting with these in amavisd-new (which calls SA) 
for the last year and more. I had to use dozens of workarounds for 
taint bugs. Luckily a usual workaround is available in a form 
of making $1 etc. local, e.g.: 
 
  { local($1,$2,$3,$4,$5,$6);  # avoid Perl 5.8.0 bug, $1 gets tainted 
    $per_msg_status = $spamassassin_obj->check($mail_obj); 
  } 
 
or: 
  { local($1,$2,$3,$4);  # avoid Perl 5.8.0..5.8.3...? taint bug 
    $spam_level  = $per_msg_status->get_hits; 
    $spam_report = $per_msg_status->get_report;  # taints $1 and $2 ! 
   } 
 
Tainting or $1, $2, ... occurs also in core Perl modules, triggered 
by certain more complex expressions (like chained s/// || s/// || ..), 
so one can never be sure that the $1 won't become tainted from an 
innocent looking operation like IO::File::open. 
 
From amavisd-new release notes, July 2004: 
- opened another can of Perl worms (taint bugs): turn on Perl pragma 
  "use re 'taint'" in all modules, and selectively turn it off where needed. 
  It replaces cumbersome manual preservation of taintedness when regexp 
  saved ranges are used without intention to untaint. Because of Perl bugs, 
  strategically placed local($1,$2,...) are needed, otherwise previous 
  taint flag in $1, $2, ... can be brought on to new variables, which can 
  all of a sudden become tainted out of nowhere; 
 
From amavisd-new release notes, April 2004: 
- when calling IO::File::open() use '+>' instead of 'w+' to avoid 
  Perl taint bug ($mode turns tainted) (bug still present in 5.8.2) 
  triggered by expression in IO::Handle::_open_mode_string(); 
 
- attack the Perl 5.8.0/5.8.1/5.8.3 taint bug (once variables $1,$2,etc 
  get tainted they start spreading taintedness to other variables): 
  * insert local($1,$2,$3,...) in blocks of code which call external 
    modules which trigger the bug (Mail::SpamAssasin, MIME::Parser, ...) 
  * insert local($1,$2,$3,...) in blocks of code which depend on these 
    variables to be clean, and which demonstrated through bug reports 
    and experience with various version of Perl that these variables 
    were not always taint-clean; 
  The last taint incident triggered by SA 3.0.0 (svn) reported by Luc de Louw 
 
My 'best practices' recommendation is to: 
- use local($1) or local($1,$2,$3,...as needed) around every regexp match 
  which produces results in these global variables; 
- use  use re 'taint'; in all modules to prevent inadvertent untaining; 
- explicitly untaint with a subroutine such as: 
 
# Return untainted copy of a string (argument can be a string or a string ref) 
sub untaint($) { 
  no re 'taint'; 
  my($str); 
  if (defined($_[0])) { 
    local($1); # avoid Perl taint bug: tainted global $1 propagates taintedness 
    $str = $1  if (ref($_[0]) ? ${$_[0]} : $_[0]) =~ /^(.*)\z/s; 
  } 
  $str; 
} 
 
 
Good luck. 
  Mark 
 
Comment 12 Loren Wilton 2005-09-05 15:50:55 UTC
This fails intermittantly with 5.8.6 also, so perl 5.8.0 isn't (the whole) 
problem.

Possibly adding the local $0 etc statements to the constructed eval strings 
would fix this.  Or would the local statement go in the routine where the 
string was evaluated?  Or in each of the procedures that has been constructed 
in the eval string?
Comment 13 Jacob Luna Lundberg 2005-10-19 19:12:18 UTC
I just wanted to add that I see this bug running SA 3.1.0 on debian stable
(using the package from unstable), and when spamd starts doing this, it also
starts doing things like:

Oct 19 09:50:12 dakar spamd[14123]: Use of uninitialized value in pattern match
(m//) at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm line 547.
Oct 19 09:50:12 dakar spamd[14123]: Use of uninitialized value in bitwise and
(&) at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm line 674.
Oct 19 09:50:12 dakar spamd[14123]: Use of uninitialized value in numeric eq
(==) at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm line 726.
Oct 19 09:50:12 dakar last message repeated 5 times
Oct 19 09:50:12 dakar spamd[14123]: Use of uninitialized value in concatenation
(.) or string at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm line 755.

and

Oct 19 09:50:12 dakar spamd[14123]: unknown type  for FROM_HOSTILE_DOMAIN: From
=~ /.*(\@|\.)yahoo\.ca.*/ at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm
line 755.

I have tried running with --round-robin and it seems to help a little but not
much.  I typically run with --max-children 3
Comment 14 David Gibbs 2005-10-27 15:35:56 UTC
FWIW: I updated to Perl 5.8.3 and SA 3.1.0, but the problem is still occurring.
 It appears to be getting worse, in fact.

The module that the error is show up in appears to be different though...

Oct 27 08:30:35 rivendell spamd[1543]: spamd: Insecure dependency in eval while
running setuid at
/usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.pm line 913. 
Comment 15 Justin Mason 2005-10-27 18:21:19 UTC
btw, just to check -- I assume you have all run "spamassassin --lint" and fixed
anything that it reports?
Comment 16 David Gibbs 2005-10-27 18:33:56 UTC
Justin: Yes, spamassassin --lint reports no errors.
Comment 17 David Gibbs 2005-10-27 19:11:26 UTC
Ok, I added a simple log line to Parser.pm just before the eval statement that
is generating the insecure dependency and got the following ...

Oct 27 12:04:43 rivendell spamd[5161]: evalstr = ("" =~ /EXECS VENTURES/i); 1;
Oct 27 12:04:43 rivendell spamd[5161]: spamd: Insecure dependency in eval while
running setuid at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/Conf/Parser.p
m line 914.

Does this help?
Comment 18 Justin Mason 2005-10-27 19:32:30 UTC
'/usr/lib/perl5/site_perl/5.8.0'

are you sure you're running 5.8.3?  what does spamassassin -D --lint report?
(it'd be worth attaching the entire output as an attachment.)
Comment 19 David Gibbs 2005-10-27 20:18:57 UTC
$perl -V
Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration:
  Platform:
    osname=linux, osvers=2.4.21-4.elsmp, archname=i386-linux-thread-multi
    uname='linux tweety.devel.redhat.com 2.4.21-4.elsmp #1 smp fri oct 3
17:52:56 edt 2003 i686 i686 i386 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686
-Dversion=5.8.3 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc
-Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
-Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads
-Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow
-Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005
-Uversiononly -Dpager=/usr/bin/less -isr -Dinc_version_list=5.8.2 5.8.1 5.8.0'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -march=i386 -mcpu=i686',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
-fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.3.3 20040412 (Red Hat Linux 3.3.3-7)',
gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.3.3.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.3.3'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic
-Wl,-rpath,/usr/lib/perl5/5.8.3/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl): 
  Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS USE_LARGE_FILES
PERL_IMPLICIT_CONTEXT
  Built under linux
  Compiled at Apr 15 2004 13:09:17
  @INC:
    /usr/lib/perl5/5.8.3/i386-linux-thread-multi
    /usr/lib/perl5/5.8.3
    /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.1/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.3
    /usr/lib/perl5/site_perl/5.8.2
    /usr/lib/perl5/site_perl/5.8.1
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.1/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.3
    /usr/lib/perl5/vendor_perl/5.8.2
    /usr/lib/perl5/vendor_perl/5.8.1
    /usr/lib/perl5/vendor_perl/5.8.0
    /usr/lib/perl5/vendor_perl
    .
Comment 20 David Gibbs 2005-10-27 20:20:24 UTC
Created attachment 3212 [details]
spamassassin -D --lint output

Output of spamassassin -D --lint for Justin
Comment 21 Jacob Luna Lundberg 2005-10-27 20:40:30 UTC
Maybe I should add that I (debian/stable) have perl 5.8.4:
"This is perl, v5.8.4 built for i386-linux-thread-multi"

On my host, where several users have custom rules, this happened at least once
per day until I set --max-conn-per-child 1 so that the spamds are not reused. 
Obviously, this defeats much of the purpose of spamd.  :-/

spamassassin --lint reports no errors running under my user.
Comment 22 Ryan Hayle 2006-01-12 06:03:42 UTC
I'm having a similar (or perhaps the same?) problem.  Using SpamAssassin
3.1.0a-1bpo1 from backports.org on Debian/sarge.

Two days ago, while I was out of town, SA suddenly stopped looking at about 1/3
of my messages, filling up my inbox very quickly.  My postfix log shows this
whenever this happens:

spamd[21737]: spamd: connection from localhost [127.0.0.1] at port 1921
spamd[21737]: spamd: setuid to x succeeded
spamd[21737]: spamd: Insecure dependency in eval while ru
nning setuid at /usr/share/perl5/Mail/SpamAssassin/Conf/Parser.pm line 913.

I am only using the default rules with one user rule in my account.  Moving this
rule into local.cf as suggested seemed to fix the problem.  This is the rule I
was using:

uri GEOCITIES_URL /geocities\.(yahoo\.)?com(\...)?\b/
describe GEOCITIES_URL Message contains URL for Geocities site
score GEOCITIES_URL    2.5

This has been working fine with no problem.  Nothing changed on my system--I
didn't upgrade anything, didn't change the rules or config files, nothing.  I
don't understand how it could start doing this all of a sudden.  I'm happy to
provide any additional information or debugging info as necessary.  If you
suspect this is a Debian bug I will file a separate bug report.  Thanks.
Comment 23 David Gibbs 2006-01-20 16:04:08 UTC
FWIW: I recently upgraded from Redhat 8 to Fedora Core 4 (with Perl 5.8.6) and
still get the error.
Comment 24 Justin Mason 2006-01-20 20:18:35 UTC
it appears this is related to either (a) additional, non-default rulesets or (b)
additional homebrew local rules.   Is anyone seeing this issue without either of
those in place?

uri GEOCITIES_URL /geocities\.(yahoo\.)?com(\...)?\b/

Ryan -- tip -- use (?:...) instead of (...), it's more efficient, and may even
help with this bug.
Comment 25 Jim Smith 2006-02-13 16:19:32 UTC
I have been plagued with this bug ever since upgrading SA to 3.1.0 on my server.
Here are my particulars if it helps. I'm running FreeBSD 4.7-RELEASE-p28. This
is a shared server and my mailbox is the only one I have detected that has this
problem (other clients mention infrequent issues with SA but so far I haven't
been able to tie it in with this bug). I run approx 2000 spams per day through
the filter. All rules worked great in version 2.?? before. No errors or anything
out of the ordinary in the --lint report. 

Due to the frustrating intermittent nature of problem, I finally set up a new
server, fresh installation of SA v 3.1.0, fresh perl v 5.8.4. Ran email for 2
days without any cusomization without incident. Put customized procmailrc file
in for a day without incident. Put in user_prefs file and it ran for approx 6 or
8 hours before the problem showed up. Same error in maillog file as others
mentioned:
spamd: Insecure dependency in eval while running setuid at
/usr/local/lib/perl5/site_perl/5.8.4/Mail/SpamAssassin/Conf/Parser.pm line 913. 
Restarted spamd and all was well for a few hours. Again restarted spamd with
similar result. Then I sat and swapped out parts of my user_prefs file. Removed
whitelist (approx 50 entries) but no change. Removed all custom rules and SA
began working again. Put back a few at a time until I was able to consistantly
replicate the problem with 2 rules:
#
body PATENT_BIZ		/based patent business/i
score PATENT_BIZ 		-2.0
describe PATENT_BIZ 	Reduce score for someone with comments about my
business
#
body YOUR_PATENT		/your patents/i
score YOUR_PATENT		-2.0
describe YOUR_PATENT	Reduce score for someone with comments about my
patents
#
Without those rules, it has been working for over 24 hours without incident. I
put those rules back in this morning, and the SA still works (this is a slippery
bug!). My guess is that if I left those rules in, after a few hours it would
again start skipping out again. 

<guess/speculation>It almost seems like a cumulative effect: I put the custom
rules in that trigger problems and all is well for a while. After a certain
threshhold, it suddenly shuts SA filters off. If I remove the problem rules, SA
immediately works. If I quickly put them back, SA shuts off. If I leave the
rules out for a period of time, whatever the problem was, clears out.

Beyond that info, I don't know what else I can offer. I hope that helps.

js
Comment 26 Daryl C. W. O'Shea 2006-02-13 16:27:55 UTC
Just to confirm...

1) These rules are in your user_prefs file and you have "allow_user_rules 1" in
your local.cf, right?

2) The problem does not occur if you put these rules in your local.cf file and
not you user_prefs file?
Comment 27 Jim Smith 2006-02-13 16:45:27 UTC
> 1) These rules are in your user_prefs file and you have "allow_user_rules 1"
in your local.cf, right?
That is correct.

> 2) The problem does not occur if you put these rules in your local.cf file and
> not you user_prefs file?

I believe that is correct. I did try putting them in the local.cf file some time
ago but didn't leave it there for too long as those rules were only specific to
me and not to be used server-wide. While they were in the local.cf file the
problem did not occur.    
Comment 28 Daryl C. W. O'Shea 2006-02-13 16:49:47 UTC
Can you try the rules (in user_prefs) without the modifiers (lose the i)?
Comment 29 David Gibbs 2006-02-13 16:50:57 UTC
FWIW: I *THINK* that the error seems to manifest more often when I have user
rules implemented.  Most of the user rules I have implemented are scanning the
subject line for various bits of text (usually case insensitive).
Comment 30 Daryl C. W. O'Shea 2006-02-13 17:00:15 UTC
Are you suggesting that you've seen the error happen when allow_user_rules was
not enabled, or when you had no rules in user_prefs (but allow_user_rules enabled)?
Comment 31 Jim Smith 2006-02-13 17:04:57 UTC
> Can you try the rules (in user_prefs) without the modifiers (lose the i)?
Ok, I made that change. As stated before it may take several hours to find out
if it makes a difference or not. I'll keep you posted.  js
Comment 32 Justin Mason 2006-02-13 19:45:32 UTC
So judging by list traffic, it now seems that everyone here is using
"allow_user_rules 1", right?  That's an important detail; speak up if not.

Also, everyone is seeing this with spamd?

does anyone see this with Amavisd-new?  (if not, maybe we should look into
copying Mark's technique.)

It's a bug in perl's taint-tracking, where it loses track of what strings are
tainted, I think possibly after lots and lots of eval '' invocations.  Normally,
without user rules, there are few of those; but with user rules, every time the
user rules file is read, they have to be compiled and an eval '' takes place, so
user rules would cause this to be more likely to occur on perl versions with the
bug. 

Setting spamd's --max-conn-per-child setting to something lower (try 50) may
reduce the likelihood of this perl bug occurring before the spamd child is recycled.
Comment 33 Loren Wilton 2006-02-14 10:29:49 UTC
For me:  Allow_user_rules, yup.  Spamd, yup.

I can not state absolutely that I've never heard of this happening *without* 
allow_user_rules; in fact I'm pretty sure I have a vague recollection of 
someone reporting a one-off that sounded suspiciously like this happening 
without allow_user_rules.  However, that was one mysterious occurance vs *lots* 
of occurances with people that allow user rules.

I've also observed that the problem seemes to run in a batch.  No problem for a 
while, then several messages in a row would not be checked, then things worked 
again for a while.  Even catching an unchecked message in procmail and sending 
it back through spamd, it would be uncaught (due to this failure) the second 
time.  However, run it through spamassassin directly and there is a very good 
chance that the spam will be correctly caught.

I was never able to tie it down to a rule since it was too sporadic.  Never 
connected it to max children or max conns, but that is a good likely 
explanation.
Comment 34 Jim Smith 2006-02-14 14:41:00 UTC
(In reply to comment #28)
> Can you try the rules (in user_prefs) without the modifiers (lose the i)?

Interesting -- I dropped the modifier on the two problematic rules and have not
had any incidents since doing so yesterday. Not 100% certain of cause/effect
because it may take a while to manifest itself but it usually would fizzle out
in a shorter time than this.

Should I put the modifiers back in, wait for SA to stop filtering and then put
them in to see if that is the trigger? Or is it more beneficial for me to leave
the rules w/o modifiers for a longer period of time to see if it stops?

js
Comment 35 Daryl C. W. O'Shea 2006-02-14 15:03:14 UTC
Please leave them out for now.  At least two or three times the maximum length
of time you've gone without failures would be a good sign.
Comment 36 Jim Smith 2006-02-15 01:19:02 UTC
(In reply to comment #35)
> Please leave them out for now.  At least two or three times the maximum length
> of time you've gone without failures would be a good sign.

After a day and a half or so of smooth operation the problem returned a few
minutes ago. I didn't do anything to trigger it -- just checked email and found
45 unscored spam in my inbox. So we now know that removing the modifier has no
bearing on the problem. So I just commented out the two problem rules mentioned
above and now it is back to working fine again.

I'd love to stay and troubleshoot it some more but it is Valentines Day and my
wife is wondering why I'm so focused on email rather than her <heh, heh>. So I
guess I'd better log off now.

js
Comment 37 Daryl C. W. O'Shea 2006-02-15 01:40:20 UTC
Thinking about it again, I guess it doesn't make too much sense that only one of
the captures would become tainted.  I just noticed that all of the rules
reported had modifiers.

Although, having fewer captures might explain the longer duration between
taintings.  At least I'd like to think that. :D
Comment 38 Joanne 2006-02-15 04:12:32 UTC
This seems ot be related to 4445. Observations there should be considered, as 
well. Here is a new observation that will require some really exhaustive and 
mind numbing log reading to prove false (from an email to Daryl.)

I went back over 4445 also. And I started looking at the logs again
both during an episode and after. This raised a really "stupid"
question. I'd have to really comb the logs to figure out in detail
if there is a counter indication. It SEEMS like the problem may come
in, for Loren and me, if fetchmail goes out and finds new mail to
queue while a previous batch is still being processed. I don't see
this processing overlap with fetchmail and spamd at times the
PerMsgStatus error message happens. At the moment mail takes, on the
average, about 3 to 4 seconds to complete spamd processing. There seem
to be times if day when there are BIG bursts of messages followed by
relative sanity the rest of the day.

So far that's the best observation I have that I've not posted before.

"A message added to the user's incoming queue while spamd is still processing
old messages leads to the hit."

Although it's on the output side rather than input side of the mail
intake process I do note we're using "mbox" format for our system here.


                /-> spamc/spamd ->\
fetchmail -> procmail---------------> /var/spool/<mbox files>

(Yes, some mails skip spamd for size or specific list reasons.)

{^_^}   Joanne
Comment 39 Jim Smith 2006-02-15 15:01:45 UTC
Created attachment 3374 [details]
My user_prefs file

I should have submitted my user_prefs file earlier as it might hold some clues.
I removed all my whitelistings but everything else is intact in my attached
user_prefs file. 

What my be of interest is that I have several customized rules that cause no
problems. It is only the two "body" rules that cause problems. When I comment
them out, the problems go away. When I comment other rules out and leave the
body ones in, the problems stay. In my experience it is only the body rules
that cause the problem. Is that because the body rules have to go through a lot
more data than the header rules? I don't know but I think there is something
that is causing those body rules to push SA over the edge.

js
Comment 40 Joanne 2006-02-18 00:28:07 UTC
Daryl provided some test code for the but. It logs all rules going into the 
raw_body and full eval strings. The results are interesting. The processing of 
rules INTO evalstr aborts at this rule:
Feb 17 01:13:16 ...: rules: testing rawbody rule: __JS_DOCWRITE at (eval 59857) 
line 2694. 
Feb 17 01:13:16 ...: error: Insecure dependency in eval while running setuid 
at /usr/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/PerMsgStatus.pm line 
2128._ , continuing 


All failures hit at this point. It is about rule 538 of some 623 rules that get 
processed on normal messages.

And now I need to clean up a 252 megabyte maillog. <sigh> I guess it'll age off 
with time.

(I also noted while doing this logging that the number of hits was down from 
normal.)

{^_-}
Comment 41 Sidney Markowitz 2006-03-09 19:23:47 UTC
Retargeting to 3.1.2 as it does not look like a release blocker.
Feel free to fix this right away for 3.1.1 if you have any idea how :)
Comment 42 Daryl C. W. O'Shea 2006-04-09 06:46:47 UTC
*** Bug 4445 has been marked as a duplicate of this bug. ***
Comment 43 Daryl C. W. O'Shea 2006-04-09 06:50:10 UTC
I can now reproduce this every few hours, or every 5,000-10,000 emails.  Taking bug.
Comment 44 Daryl C. W. O'Shea 2006-04-12 02:07:12 UTC
Created attachment 3469 [details]
patch

Sending        lib/Mail/SpamAssassin/Conf/Parser.pm
Transmitting file data .
Committed revision 393358.
[dos@silver trunk]$
Comment 45 Daryl C. W. O'Shea 2006-04-12 02:11:10 UTC
This patch has prevented the problem from happening after processing 55,000
messages.  Normally it would happen after 5,000 messages and no more than 9,500.

Please review this one line addition for 3.1.2.
Comment 46 Joanne 2006-04-12 04:32:33 UTC
Might this patch also be back portable to the 3.0.5 build?
Comment 47 Daryl C. W. O'Shea 2006-04-12 04:38:00 UTC
(In reply to comment #46)
> Might this patch also be back portable to the 3.0.5 build?

There's nothing to back port... the (whole 1 line) patch applies cleanly to 3.0.
Comment 48 Justin Mason 2006-04-12 10:46:48 UTC
that was it?!  brilliant ;)

+1.

anyone else running into this -- please let us know if it helps for your
situation, too...
Comment 49 David Gibbs 2006-04-12 12:22:19 UTC
Although the general level of spam I recieive has dropped due to greylisting, I
applied this patch and enabled some user rules and still receive the error ...

Apr 12 04:46:57 rivendell spamd[29896]: spamd: Insecure dependency in eval while
running setuid at
/usr/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/PerMsgStatus.pm line 2205,
<GEN40> line 107. 

... and some spam is slipping through totally untagged. :(
Comment 50 David Gibbs 2006-04-12 12:32:22 UTC
(In reply to comment #49)
> Although the general level of spam I recieive has dropped due to greylisting, I
> applied this patch and enabled some user rules and still receive the error ...

Um, hold off a bit on this ... I may have patched the wrong module in SA.
Comment 51 Sidney Markowitz 2006-04-12 20:40:37 UTC
+1

The patch is good even if comment #49 turns out to not be a false alarm and we
have to keep looking for more causes.
Comment 52 Daryl C. W. O'Shea 2006-04-12 21:25:34 UTC
(In reply to comment #48)
> that was it?!  brilliant ;)

$pat was still tainted when each of the evals were being constructed after
~5,000+ messages when a single "full" user rule was in use.  Tracking it back to
what was supposed to untaint it led to what Mark brought up in comment 11.


Sending        lib/Mail/SpamAssassin/Conf/Parser.pm
Transmitting file data .
Committed revision 393590.
[dos@silver 3.1]$


David -- please let us know if it turns out to work for you... I believe it should.
Comment 53 Joanne 2006-04-13 00:04:10 UTC
Note that this problem was not restricted to "full". I believe it affected two 
or three other types of evalstr rules. If this patch does not address a common 
point for all of them then it is perhaps "good but insufficient."

{^_^}
Comment 54 Daryl C. W. O'Shea 2006-04-13 00:31:09 UTC
(In reply to comment #53)
> Note that this problem was not restricted to "full". I believe it affected two 
> or three other types of evalstr rules. If this patch does not address a common 
> point for all of them then it is perhaps "good but insufficient."

Yes, "full" was just an example of how to reproduce.  The patch fixes the
problem with any standard rule type, dammit! :)

Comment 55 David Gibbs 2006-04-14 13:20:44 UTC
(In reply to comment #50)
> Um, hold off a bit on this ... I may have patched the wrong module in SA.

It appears to be OK now ... I guess I did patch the wrong version of the
Paser.pm module. 

Comment 56 Kenneth Porter 2006-05-22 22:55:38 UTC
I've been running about 45 minutes without sign of recurrence of this error, and
I've been seeing 2-3 per hour before this.
Comment 57 Daryl C. W. O'Shea 2006-05-23 04:24:58 UTC
(In reply to comment #56)
> I've been running about 45 minutes without sign of recurrence of this error, and
> I've been seeing 2-3 per hour before this.

Thanks for the report Ken!