Bug 7629

Summary: Use of uninitialized value $(
Product: Spamassassin Reporter: serhioz <serhioz>
Component: spamc/spamdAssignee: SpamAssassin Developer Mailing List <dev>
Status: RESOLVED WONTFIX    
Severity: blocker CC: billcole, cepheid, davej, gfa, kmcgrail, marcel.evenson, serhioz
Priority: P2    
Version: 3.4.2   
Target Milestone: 3.4.3   
Hardware: PC   
OS: Linux   
Whiteboard:
Attachments: Patch to handle $( pedantically
Revised patch

Description serhioz 2018-09-19 12:13:22 UTC
Centos 6.
SpamAssassin 3.4.2 from cpan
I have warnings in log

spamd[1194]: Use of uninitialized value $( in numeric ne (!=) at /usr/local/share/perl5/Mail/SpamAssassin/Util.pm line 1510.
spamd[1194]: Use of uninitialized value $( in concatenation (.) or string at /usr/local/share/perl5/Mail/SpamAssassin/Util.pm line 1595.
spamd[1194]: util: setuid: ruid=498 euid=498 rgid= 498 egid=498 93 498 

while running pyzor client from spamd.

At the same time running 
$ spamassassin -D <123.txt
from user shell is okay

[2108] dbg: pyzor: opening pipe: /usr/bin/pyzor check < /var/amavis/tmp/.spamassassin2108ot9rE8tmp
[2123] dbg: get_user_groups: uid is 498
[2123] dbg: get_user_groups: added 93 (exim) to group list which is now: 498 93 
[2123] info: util: setuid: ruid=498 euid=498 rgid=498 93 498 egid=498 93 498 
[2108] dbg: pyzor: [2123] finished: exit 1
[2108] dbg: pyzor: got response: public.pyzor.org:24441 (200, 'OK') 0 0
Comment 1 serhioz 2018-09-19 14:48:56 UTC
adding -g groupname to spamd does not solve it.
Comment 2 Kevin A. McGrail 2018-09-21 11:39:44 UTC
It looks like get_user_groups is not always working in Util.pm

Please add some debug output
Comment 3 serhioz 2018-09-21 11:47:06 UTC
-D switch

spamd[24866]: get_user_groups: uid is 498
spamd[24866]: get_user_groups: added 93 (exim) to group list which is now: 498 93 
spamd[24866]: Use of uninitialized value $( in numeric ne (!=) at /usr/local/share/perl5/Mail/SpamAssassin/Util.pm line 1510.
spamd[24866]: Use of uninitialized value $( in concatenation (.) or string at /usr/local/share/perl5/Mail/SpamAssassin/Util.pm line 1595.
spamd[24866]: util: setuid: ruid=498 euid=498 rgid= 498 egid=498 93 498 


# LANG=C;id amavis
uid=498(amavis) gid=498(amavis) groups=93(exim),498(amavis)

looks like special variable $( not defined in spamd daemon mode (-d)
Comment 4 Bill Cole 2018-09-21 20:54:06 UTC
Created attachment 5601 [details]
Patch to handle $( pedantically

I can't reproduce the error on my test systems (which don't include CentOS 6) and it is perverse that the $( variable could *ever* be uninitialized: it's part of the Perl runtime. The fact that this has only been reported for CentOS 6 makes me suspect that it is an artifact of the Perl version. The 'delta' document for 5.16.0 includes a note on no longer "caching" the values of $<, $>, $(, and $) (from when?) because they could "drift out of sync with reality." 

Obviously we still need to deal with this oddity, even if ultimately it is a bug in Perl 5.10.

One apparent bit of sloppiness in the code is that it treats $( as a simple numeric value, which it is not. It is a space-delimited list of numbers. That apparently is supposed to be handled automagically by perl but the attached patch does it in explicit code and can't hurt. It is unlikely to fix the reported problem, but since I have not reproduced that I guess it's possible...
Comment 5 serhioz 2018-09-23 18:09:07 UTC
thanks, it works.
what about line 1595?

  info("util: setuid: ruid=$< euid=$> rgid=$( egid=$) ");

using split on that line there are still errors

 Use of uninitialized value $( in concatenation (.) or string 
 Use of uninitialized value $) in concatenation (.) or string
Comment 6 Bill Cole 2018-09-25 14:27:59 UTC
(In reply to serhioz from comment #5)
> thanks, it works.

I find that a bit surprising... 

> what about line 1595?
> 
>   info("util: setuid: ruid=$< euid=$> rgid=$( egid=$) ");
> 
> using split on that line there are still errors
> 
>  Use of uninitialized value $( in concatenation (.) or string 
>  Use of uninitialized value $) in concatenation (.) or string

I'm stumped. I can see how those variables might be wrong, but "uninitialized" makes no sense.

I've stood up a CentOS 6.10 test VM but have been unable to reproduce the errors. It may be an issue specific to how you are running spamd.
Comment 7 serhioz 2018-09-25 14:36:37 UTC
maybe, there is my user in two groups

uid=498(amavis) gid=498(amavis) groups=93(exim),498(amavis)

add your test user to one another group
Comment 8 Dave Jones 2018-09-25 15:45:59 UTC
I am getting this same error message on C6 using MailScanner so it's not just spamd or amavis.  The same configs, rules and Bayes in redis do not give this error on C7's version of perl.
Comment 9 Marcel 2018-10-04 11:51:56 UTC
I did up some test RPMs for Spamassassin 3.4.2 on Centos 7 and I'm seeing the same thing (spamd is running with --daemonize). With Bill Cole's patch applied:

Oct  3 23:19:05 el7p17 spamd[36364]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595.
Oct  3 23:19:05 el7p17 spamd[36364]: Use of uninitialized value $) in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595.
Oct  3 23:19:05 el7p17 spamd[36364]: util: setuid: ruid=30 euid=30 rgid= 984 31 984 egid= 984 31 984
Comment 10 Marcel 2018-10-04 11:55:59 UTC
Sorry more info on my test system:

# cat /etc/sysconfig/spamassassin
# Options to spamd
SPAMDOPTIONS="--create-prefs --daemonize --helper-home-dir=/var/qmail --max-children=5 --nouser-config --username=popuser --virtual-config-dir=/var/qmail/mailnames/%d/%l/.spamassassin"

# cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 

# perl -v
This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi(with 33 registered patches, see perl -V for more detail)
Comment 11 Amir Caspi 2019-03-25 23:03:22 UTC
(In reply to Marcel from comment #9)
> I did up some test RPMs for Spamassassin 3.4.2 on Centos 7 and I'm seeing
> the same thing (spamd is running with --daemonize).

I'm seeing the line 1595 error as well, on a brand-new CentOS 7 system with SA 3.4.2 installed via FC27 SRPM (since EPEL is lagging behind).  Using spamd per-user via procmail (no amavisd):

[root@server ~]# grep uninitialized /var/log/maillog
# NOTE: these are excerpts, not all log lines shown
Mar 24 03:28:30 server spamd[27147]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595, <GEN15> line 55.
Mar 24 03:28:35 server spamd[27149]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595.
Mar 25 03:45:36 server spamd[11238]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595, <GEN13> line 55.


System info:

[root@server ~]# cat /etc/procmailrc 
:0 fw
| /usr/bin/spamc -u "${LOGNAME}" -s 1500000

[root@server ~]# cat /etc/sysconfig/spamassassin 
# Options to spamd
SPAMDOPTIONS="--max-conn-per-child=1 -c -m5 -H"

[root@server ~]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 

[root@server ~]# perl -v
This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi
(with 39 registered patches, see perl -V for more detail)


Anything I can check to help debug this?
Comment 12 Amir Caspi 2019-03-26 07:58:55 UTC
I'll note that I see this error with my 'centos' user, who is in four different groups (though only two show up in the log for some reason):

util: setuid: ruid=1000 euid=1000 rgid= 10 190 egid= 10 190

With spamd running for different users who are only in a single group, I do not get this message:

util: setuid: ruid=22003 euid=22003 rgid=505 egid=505

Would a split help here, too?
Comment 13 Bill Cole 2019-04-09 03:00:32 UTC
Created attachment 5649 [details]
Revised patch

This patch revision is illuminated by Amir Caspi's debug info posted to the mailing list
Comment 14 Bill Cole 2019-04-09 21:34:40 UTC
I reproduced the bug on a CentOS 7 machine with the stock Perl (5.16.3) and unmodified SA 3.4.2 code. It is occurring when the Pyzor plugin launches a helper process and needs a safe pipe to it, in pyzor_report() calling M:SA:Util->helper_app_pipe_open() which calls setuid_to_euid(). In my testing I found that $) was already broken in M:SA:Plugin::Pyzor->pyzor_report() so there's not much point in continuing to troubleshoot in M:SA:Util. 

Given the strange breakage (apparently the first element in $( and $) getting silently replaced by a null?) I was pretty sure that this was a problem in Perl itself, not a consequence of SA code. To test this, I built an alternative perl (5.18.4) with default build settings and built SA against it. This DID NOT exhibit the same problem. It also is not reproducible on my personal system (MacOS 10.6.8 Perl 5.26.) I believe this in dictates it to be a problem with the RH perl build, which uses '-D_FORTIFY_SOURCE=2' and has been problematic before (see bug 7591.)

The workarounds for this *cosmetic* issue are:

1. Build and use a bespoke Perl without GCC's '-D_FORTIFY_SOURCE=2' flag.
2. Comment out the line causing this (it's just informational logging anyway...) 
3. Disable Pyzor
4. Run on a different distribution family of Linux (e.g. Debian/Ubuntu, SuSE, etc.) or on a different OS.
Comment 15 Amir Caspi 2019-04-09 23:00:23 UTC
Thanks for the detective work, Bill.

For the record: I get the same error when trying to use the English variable aliases $GID/$EGID, as well.  I figured as much but thought it was worth testing... same result.

Workaround 3 may not be sufficient -- this error occurs twice for me, I think the first time is when dccproc opens the helper.  So one would have to disable both DCC and Pyzor.  That seems like a rather extreme solution to a cosmetic problem, however... I don't think I'd recommend this particular workaround regardless. =)

I would have normally suggested that I should file an upstream bug report at RH but from bug #7591 is seems like they want a clean snippet that reproduces this... and it sounds like that's not something easy to make.  Unless you have an easy snippet that reproduces this, from your testing?

Should I file a bug report upstream anyway, since this particular bug may have other implications beyond SA?  Or is it pointless?
Comment 16 Bill Cole 2019-04-09 23:27:03 UTC
Note that any one of those workarounds would be adequate in itself. And yes, DCC would also hit this bug. I'm tempted to just remove that logging line from Util.pm and call it a fix, but someone at some point thought the information was useful and I'm not entirely comfortable with removing diagnostic code just because it breaks. 

I have no simplified self-contained code for demonstrating this issue. I don't really know what RH would do with it but I suppose it couldn't hurt to open a bug report with them, since it's their Perl build acting up.