Bug 45494

Summary: testsockets fails on Solaris when IPv6 interfaces are not configured
Product: APR Reporter: Mikhail T. <mi+apache>
Component: APR testAssignee: Apache Portable Runtime bugs mailinglist <bugs>
Status: REOPENED ---    
Severity: normal CC: bojan
Priority: P3    
Version: HEAD   
Target Milestone: ---   
Hardware: PC   
OS: Solaris   

Description Mikhail T. 2008-07-28 14:22:15 UTC
I built apr-1.3.2 fresh on Solaris-10. Using the following configure:

env CC=cc CFLAGS="-xarch=v9b -fast" CXXFLAGS="-xarch=v9b -fast" ./configure --with-sendfile --with-devrandom --prefix=/apps/foo

The build completed cleanly, but two of the tests failed:

[...]
testsleep           : SUCCESS
testshm             : SUCCESS
testsock            : FAILED 1 of 8
testsockets         : FAILED 1 of 7
testsockopt         : SUCCESS
teststr             : SUCCESS
teststrnatcmp       : SUCCESS
testtable           : SUCCESS
testtemp            : SUCCESS
testthread          : SUCCESS
testtime            : SUCCESS
testud              : SUCCESS
testuser            : SUCCESS
testvsn             : SUCCESS
Failed Tests            Total   Fail    Failed %
===================================================
testsock                    8      1     12.50%
testsockets                 7      1     14.29%
*** Error code 1

Please, advise. Thanks!
Comment 1 Bojan Smojver 2008-08-08 22:09:27 UTC
I'm not a Solaris guy, but I kinda remember problems with these tests when no IPv6 interfaces were configured on the machine. Do you have IPv6 configured on this system?
Comment 2 Mikhail T. 2008-08-10 21:10:25 UTC
No, there are no IPv6 interfaces. Here is the output of "ifconfig -a":

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000 
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.77.11.220 netmask ffffff00 broadcast 10.77.11.255
Comment 3 Ruediger Pluem 2008-08-10 23:52:20 UTC
Thats likely the reason why it fails. Create an ipv6 loopback interface and things should work:
ifconfig lo0 inet6 plumb
ifconfig lo0 inet6 ::1 up
Comment 4 Rainer Jung 2008-08-11 06:23:30 UTC
If you run "testall -v" in the test sub directory, you will get additional information concerning the failure reason.

The failure of testsock should be fixed by now, see

http://svn.apache.org/viewvc?rev=684617&view=rev

Here the test case was not really valid.

The failure of testsockets was reproducible here and fixed by RĂ¼diger's recommendation on how to configure an IPv6 interface with address ::1.
Comment 5 Mikhail T. 2008-08-11 08:34:50 UTC
Here are the two tests' verbose output:

testsock            : |Line 92: Problem getting http service (2): No such file or directory

|Line 290: Cannot test if connect completes synchronously
FAILED 1 of 8
testsockets         : /Line 131: Could not bind socket (126): Cannot assign requested address

-Line 185: Condition is false, but expected true
FAILED 1 of 7
Comment 6 Bojan Smojver 2008-08-11 16:50:04 UTC
Closing. The problem with http service was fixed and the failure related to IPv6 can be avoided by configuring an IPv6 interface on the system.
Comment 7 Mikhail T. 2008-08-12 07:27:53 UTC
With all due respect, having IPv6 configured should not be a requirement for the test's passing. In fact, having /any/ networking configured should not be a requirement.

If IPv6 (or v4) is not set up, the test should detect that and skip quietly -- without a false negative.

If the tests are known to give false alarms, a real alarm will be mistaken for a false one -- arp is the foundation of many software packages, it better be /squeaky/ clean :-)
Comment 8 Bojan Smojver 2008-08-12 07:39:03 UTC
All right, fair enough. I changed the name of the bug to reflect what's still broken.
Comment 9 William A. Rowe Jr. 2008-08-12 09:31:50 UTC
Sorry, but re-closing.

IF IPV6 networking is not installed, the tests /are/ clean.

If IPv6 interfaces are not configured for an IPv6 machine, the tests
rightfully fail.  This is true because such a misconfigured system would
not perform these unit tests, and therefore (under your suggestion) may
provide a false positive for success where the code may be faulty.
Comment 10 Mikhail T. 2008-08-12 09:40:30 UTC
Absence of IPv6 is NOT a misconfiguration... Neither is the lack of IPv4.

The test's status in this case should be "SKIPPED". "FAILURE" is worse, as it raises a false alarm. A "SUCCESS" would be misleading (false positive), but not as bad.

If adding the third state ("SKIPPED") requires changes to the entire test-harness -- please, rename/reclassify this bug.

But the simple rule should be: "run `make check' after building". If you want people to do this, the tests should not raise false alarms, even if their coverage may sometimes be incomplete -- is it ever, really?

Nobody will blame you for NOT testing IPv6 on a machine, that does not have it, but FAILING a test of a piece, that's not enabled on a system, is ridiculous.
Comment 11 William A. Rowe Jr. 2008-08-12 10:06:22 UTC
You are missing the point, you installed IPv6 on this machine.  You did not
configure an interface.

Either; 1) install IPv4 layer only, and you have nothing more to do, or...
2) install an IPv6 loopback device, to complete the installation of IPv6.

If you build for IPv6, the ./testall cannot report success without validating
certain exception conditions which vary from platform to platform, and as-yet
undiscovered flaws on future platform/releases.

If you want to have an apr which doesn't test or validate ipv6, you can do so;

  ./configure --disable-ipv6

If you are concerned that IPv6 networking components are installed by Solaris 10
without configuring a loopback endpoint, contact Sun.

If you are concerned that we should pass these tests for IPv6 in spite of this
misconfiguration, it would stand to reason that we would have to give a pass to
any misconfigured IPv4-only machine as well.  The purpose of the test suite is
to validate all enabled apr interfaces, so the first half of this bug report
was 'FIXED', the second half (and current title) is INVALID.
Comment 12 Mikhail T. 2008-08-12 10:13:09 UTC
= you installed IPv6 on this machine.  You did not configure an interface.

I did not install anything on this machine -- it came from Sun preinstalled with the standard Solaris 10. Having a configured IPv6, when there is no use for it is a waste, so I agree with Sun, that it is useless to enabled it.

The test should not have reported a failure. It should've SKIPPED.

= ./configure --disable-ipv6

Yeah, that's a work-around. Except I may wish to -- some day -- be able to use it with IPv6 too. Telling me, that a test was SKIPPED because IPv6 is not currently in use on the testing machine, is Ok.

Telling me, the test FAILED is bogus...

This is all patently obvious, I might add... Even IF the machine is misconfigured, the test should not "FAIL", because there is nothing wrong with the build.
Comment 13 Mikhail T. 2012-03-26 21:25:42 UTC
Still a problem... Solaris 10, building apr-1.4.6:

testatomic          :  SUCCESS
testdir             :  SUCCESS
testdso             :  SUCCESS
testdup             :  SUCCESS
testenv             :  SUCCESS
testfile            :  SUCCESS
testfilecopy        :  SUCCESS
testfileinfo        :  SUCCESS
testflock           :  SUCCESS
testfmt             :  SUCCESS
testfnmatch         :  SUCCESS
testargs            :  SUCCESS
testhash            :  SUCCESS
testipsub           :  SUCCESS
testlock            :  SUCCESS
testcond            :  SUCCESS
testlfs             :  SUCCESS
testmmap            :  SUCCESS
testnames           :  SUCCESS
testoc              :  SUCCESS
testpath            :  SUCCESS
testpipe            :  SUCCESS
testpoll            :  SUCCESS
testpools           :  SUCCESS
testproc            :  SUCCESS
testprocmutex       :  
SUCCESS
testrand            :  SUCCESS
testsleep           :  SUCCESS
testshm             :  SUCCESS
testsock            :  SUCCESS
testsockets         :  FAILED 1 of 7
testsockopt         :  SUCCESS
teststr             :  SUCCESS
teststrnatcmp       :  SUCCESS
testtable           :  SUCCESS
testtemp            :  SUCCESS
testthread          :  SUCCESS
testtime            :  SUCCESS
testud              :  SUCCESS
testuser            :  SUCCESS
testvsn             :  SUCCESS
Failed Tests            Total   Fail    Failed %
===================================================
testsockets                 7      1     14.29%

Rehashing this 4-year old argument, that the build-machine does not have an IPv6 interface should not cause the test to fail. The test's verdict may be "skipped", but it should not be a "fail".

Using the ``--disable-ipv6'' switch wish configure may be a work-around for some, but not for all -- the machine(s), where the compiled binaries will be used may well have/need IPv6 interfaces.