Bug 6286 - [review] make test for rc3 fails (sporadically?) for timeout.t test
Summary: [review] make test for rc3 fails (sporadically?) for timeout.t test
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Regression Tests (show other bugs)
Version: 3.3.0
Hardware: Other All
: P1 blocker
Target Milestone: 3.3.0
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard: ready for commit
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-15 10:10 UTC by Kevin A. McGrail
Modified: 2010-01-15 12:04 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
proposed patch to the t/timeout.t test patch None Mark Martinec [HasCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin A. McGrail 2010-01-15 10:10:08 UTC 
Mark Martinec Notes:
>> It can fail on a heavily loaded machine, as it is a
>> real-time test. Before opening a bug, try it again.

Warren Togami Notes:
I saw something like this while cutting rc1.  The test failed once, but 
I wasn't able to reproduce the test failure in dozens of subsequent test 
runs.

Phil Randall Notes:
I had the same problem on a test CentOS 5.4 VM with with Warren's 3.3.0
rc3 yesterday.

I was able to duplicate a failure on test 16 2x in a run. Running a third now...

t/timeout.........................ok 15/33# Failed test 16 in t/timeout.t at 
line 99
t/timeout.........................FAILED test 16
        Failed 1/33 tests, 96.97% okay
Comment 1 Kevin A. McGrail 2010-01-15 10:17:02 UTC 
OK, good news, my machine failed 3x's on timeout.t with net and long tests.  

Now I've also modified Makefile to run ONLY t/timeout and still fail.

I then compiled as a normal user instead of root and modified the makefile as well to run only the t/timeout.t for a make test.  

Still failing at line 99
t/timeout....ok 15/33# Failed test 16 in t/timeout.t at line 99  

That means I have a box we can reproduce the issue.
Comment 2 Mark Martinec 2010-01-15 10:35:32 UTC 
Created attachment 4643 [details]
proposed patch to the t/timeout.t test

The test on a reset() indeed had a close margin and could fail,
because of ditching fractions when a new alarm() is placed in a reset(),
so it could just make it, or just not make it.

The patch increases too tight margins on this test #16, and also on other
tests, so that margins are now 2 seconds, instead of 1 as before
(previously 0 on a reset test).
Comment 3 Mark Martinec 2010-01-15 10:38:43 UTC 
Btw, no need to rerun the complete 'make test' every time,
$ prove t/timing.t    suffices.
Comment 4 Kevin A. McGrail 2010-01-15 11:07:17 UTC 
Thanks re: prove.  I had been trying to make sure it wasn't a heavily loaded machine that was leading to the problem.

+1 on this patch.  timeout appears resolved.

KAM
Comment 5 Warren Togami 2010-01-15 11:19:23 UTC 
+1 to patch, even though I can't make it fail without the patch.
Comment 6 Mark Martinec 2010-01-15 11:26:19 UTC 
  Bug 6286: relax timing margins on the t/timeout.t test,
  especially on the #16 reset() test, which was too tight
Sending t/timeout.t
Committed revision 899774.
Comment 7 Mark Martinec 2010-01-15 12:04:25 UTC 
closing