Bug 7181 - t/sa_compile.t fails make disttest but passes with prove -v t/sa_compile.t
Summary: t/sa_compile.t fails make disttest but passes with prove -v t/sa_compile.t
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Regression Tests (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: PC Windows 7
: P2 blocker
Target Milestone: 3.4.2
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-04-28 06:56 UTC by Kevin A. McGrail
Modified: 2017-04-11 18:54 UTC (History)
5 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin A. McGrail 2015-04-28 06:56:28 UTC
So I decided to test the release with re2c.  That led to an error with make disttest for the sa_compile.t, specifically:

cannot write /usr/src/Mail-SpamAssassin/trunk/Mail-SpamAssassin-3.4.1/t/log/d.sa_compile/inst.basic/foo/share/spamassassin/20_testrules.cf at t/sa_compile.t line 78.

I added a bit more error checking and path creation:

Index: SATest.pm
===================================================================
--- SATest.pm   (revision 1676407)
+++ SATest.pm   (working copy)
@@ -166,7 +166,9 @@
   # do not remove prior test results!
   # rmtree ("log");
 
-  mkdir ("log", 0755);
+  unless (-d "log") {
+    mkdir ("log", 0755) or die ("Error creating log dir: $!");
+  }
   chmod (0755, "log"); # set in case log already exists with wrong permissions
   system("chacl -B log 2>/dev/null || setfacl -b log 2>/dev/null"); # remove acls that confuse test
 
Index: sa_compile.t
===================================================================
--- sa_compile.t        (revision 1676407)
+++ sa_compile.t        (working copy)
@@ -5,6 +5,8 @@
 use SATest; sa_t_init("sa_compile");
 use Test;
 use Config;
+use File::Basename;
+use File::Path qw/mkpath/;
 
 my $temp_binpath = $Config{sitebinexp};
 $temp_binpath =~ s/^\Q$Config{prefix}\E//;
@@ -75,8 +77,14 @@
 
 sub set_rules {
   my $rules = shift;
-  open RULES, ">$instdir/foo/share/spamassassin/20_testrules.cf"
-          or die "cannot write $instdir/foo/share/spamassassin/20_testrules.cf";
+
+  #Create the dir for the cf file
+  my $file = "$instdir/foo/share/spamassassin/20_testrules.cf";
+  my $dir = dirname($file);
+  mkpath($dir);
+
+  open RULES, ">$file"
+          or die "cannot write $file - $!";
   print RULES qq{
 
     use_bayes 0
@@ -86,8 +94,13 @@
   };
   close RULES or die;
 
-  open RULES, ">$instdir/foo/etc/mail/spamassassin/v330.pre"
-          or die "cannot write $instdir/foo/etc/mail/spamassassin/v330.pre";
+  #Create the dir for the pre file
+  $file = "$instdir/foo/etc/mail/spamassassin/v330.pre";
+  $dir = dirname($file);
+  mkpath($dir);
+
+  open RULES, ">$file"
+          or die "cannot write $file - $!";
   print RULES qq{
 
     loadplugin Mail::SpamAssassin::Plugin::MIMEHeader

However, that still left me with this:

t/sa_compile.t .................... 1/? # Failed test 1 in t/sa_compile.t at line 149
        Not found: FOO =  check: tests=FOO  at t/sa_compile.t line 150.
# Failed test 2 in t/SATest.pm at line 755
Can't exec "/usr/src/Mail-SpamAssassin/trunk/Mail-SpamAssassin-3.4.1/t/log/d.sa_compile/inst.basic/foo//bin/sa-compile": No such file or directory at t/SATest.pm line 990.
'/usr/src/Mail-SpamAssassin/trunk/Mail-SpamAssassin-3.4.1/t/log/d.sa_compile/inst.basic/foo//bin/sa-compile --keep-tmps' failed: DIED, signal 127 (ffffffff) at t/SATest.pm line 991.
t/sa_compile.t .................... Dubious, test returned 2 (wstat 512, 0x200)
Failed 5/5 subtests 

Since sa_compile.t still passes otherwise with make test or prove -t t/sa_compile, I'm pushing this to 3.4.2 since it's likely been in this state for ages.
Comment 1 Kevin A. McGrail 2015-04-28 07:20:41 UTC
Moving this to a blocker for 3.4.2 so it gets some attention and committing changes so far on the issue.

As a side note, does make test and make disttest run the same tests?  Something to answer when this is done.

svn commit -m 'work on bug 7181 for make disttest and sa_compile.t'
Sending        t/SATest.pm
Sending        t/sa_compile.t
Transmitting file data ..
Committed revision 1676422.
Comment 2 Kevin A. McGrail 2015-04-28 07:30:48 UTC
important note for the push to 3.4.2, the xt/50_sa_compile.t passes as well.
Comment 3 Joe Quinn 2015-04-28 14:57:57 UTC
+1 for targeting 3.4.2, unless someone who's actually using it can prove there's a real problem. It doesn't look like sa-compile has changed significantly in quite a while, so this must be an old issue and one more release won't hurt.
Comment 4 John ffitch 2015-05-01 15:06:53 UTC
I have a version of this error except with proove -v t/sa-compile.t also fails so I am not really happy installing 3.4.1
  Has been OK on earlier releases

Debian 7.4 64bit on Intel hardware
Comment 5 Kevin A. McGrail 2015-05-01 15:12:06 UTC
(In reply to John ffitch from comment #4)
> I have a version of this error except with proove -v t/sa-compile.t also
> fails so I am not really happy installing 3.4.1
>   Has been OK on earlier releases
> 
> Debian 7.4 64bit on Intel hardware

Please note if you are not using compiled rules, you have no worries.

Hopefully we can get to the bottom of this failure!
Comment 6 Kevin A. McGrail 2015-05-01 17:12:19 UTC
Continuing the discussion:

John Ffitch confirmed on the mailing list on his system that the test passes with 3.4.0 but fails on 3.4.1

The bulk of the changes for that are from this commit: http://mail-archives.apache.org/mod_mbox/spamassassin-commits/201402.mbox/%3C20140220194548.461FD23888E2@eris.apache.org%3E

from this bug https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7005 might be a culprit.

I've asked him to copy the sa_compile from 3.4.0 to 3.4.1 and try that test.

He reported that "Copying t/sa_compile.t from 3.4.0 to 3.4.1 and then prove runs OK" so we do have some issues with bug 7005.  Reopening that bug.



As a side note, I can confirm that make disttest with re2c installed fails on my test system on 3.4.0.  This is the basis for why I declared this wasn't a new issue and had more to do with the installation of re2c on my box bringing to light a hidden issue:

t/sa_compile.t .................... cannot write /usr/src/Mail-SpamAssassin/Mail-SpamAssassin-3.4.0/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/share/spamassassin/20_testrules.cf at t/sa_compile.t line 74.
t/sa_compile.t .................... Dubious, test returned 2 (wstat 512, 0x200)
Failed 5/5 subtests 


However, it does pass in 3.3.2.  So food for thought while we try and figure out why make disttest fails for this test.
Comment 7 Kevin A. McGrail 2015-05-07 00:54:58 UTC
More Updates:

I was wrong.  make disttest does NOT not work on my test rig with v3.3.2.  config.dist on 3.3.2 has long tests disabled and I did not pay close enough attention having forgotten when we changed that default.

The Error is t/sa_compile.t .................... cannot write /usr/src/Mail-SpamAssassin/Mail-SpamAssassin-3.3.2/Mail-SpamAssassin-3.3.2/t/log/d.sa_compile/inst.basic/foo/share/spamassassin/20_testrules.cf at t/sa_compile.t line 74.


I also then checked 3.3.0 as well when I believe sa_compile was added and the "good news" is that make disttest also fails for 3.3.0 t/sa_compile.t .................... cannot write /usr/src/Mail-SpamAssassin/Mail-SpamAssassin-3.3.0/Mail-SpamAssassin-3.3.0/t/log/d.sa_compile/inst.basic/foo/share/spamassassin/20_testrules.cf at t/sa_compile.t line 74.

So this issue has been here a long time and my recent installation of re2c + config.dist enabling long tests by default brought it to light.


Can't exec "/usr/src/Mail-SpamAssassin/3.4/Mail-SpamAssassin-3.4.2/t/log/d.sa_compile/inst.basic/foo/bin/sa-compile": No such file or directory at t/SATest.pm line 990.


So looking at recreating why it passes prove but not make test gets into the $(PASTHRU) variables:

On my test rig, a make disttest runs distdir, then cd's to that distdir, runs make and then runs make test LIBPERL_A="libperl.a" LINKTYPE="dynamic" PREFIX="/usr/local" recreates the failure.

That's good because I can now recreate the issue locally faster.  My plan is to fix sa_compile.t to run on my test rig and then if needed, added to config.dist another option specific to sa_compile.  After that, I'll update build/README so that if it works on the test rig, we can perform a release without having people in the wild have run this test which is a bit odder than all the other tests and has issues with prefix issues.
Comment 8 Kevin A. McGrail 2017-03-12 14:58:56 UTC
As discussed from building an RC weekly, it is a known deficiency that sa_compile.t won't pass at this time.
Comment 9 Sidney Markowitz 2017-04-11 15:01:43 UTC
It was using perl's install prefix instead of the spamassassin install prefix, so when it should have stripped /usr/local from /usr/local/bin it only stripped off the /usr. Also it left an extra /.

Committed revision 1790984.

By the way, I was never able to get this test to work on a Mac because Mac will not install Encode::Detect which is needed for this test. While working on this I stumbled on this patch to perl.h that allows Encode::Detect to install on a Mac. I want to document it in this comment so it doesn't get totally lost.

https://trac.macports.org/attachment/ticket/43150/perl5.16-dNOOP.diff
Comment 10 Kevin A. McGrail 2017-04-11 16:31:39 UTC
For 3.4 Committed revision 1790998.
Comment 11 Sidney Markowitz 2017-04-11 18:54:45 UTC
Re the comment I posted about Encode::Detect on Mac OS: I checked perl sources and see that it is fixed as of perl 5.17.1, so nothing needs to be done to have Encode::Detect and this test work on Mac OS other than have a version of perl newer than that.