This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 247704

Summary:	PHPUnit tests fail to execute tests for WordPress project that work from command line
Product:	php	Reporter:	rweaver <rweaver>
Component:	PHPUnit	Assignee:	Tomas Mysik <tmysik>
Status:	RESOLVED FIXED
Severity:	normal	CC:	rweaver, vriha
Priority:	P3
Version:	8.0.2
Hardware:	All
OS:	All
Issue Type:	DEFECT	Exception Reporter:
Attachments:	Dialog for PHPUnit configuration

Description rweaver 2014-10-06 00:23:48 UTC

It appears there is something wrong with the NetBeansSuite.php code.

Problem creation:

1. Download the WordPress project from https://develop.svn.wordpress.org/trunk
2. Run the phpunit tests from the command line and you'll see some failures.
3. Run the tests from NetBeans, and I get "no tests executed"
4. run the same command from the command line and it also runs no tests.
5. remove the NetBeansSuite.php from the command line and it runs the tests.

See: http://screencast.com/t/nKHONHSLSSH showing NB and command line as copied from the output window being run in the project folder.

Output from command line running the command:

Last login: Sun Oct  5 18:12:25 on ttys000
server:~ robweaver$ phpunit
PHPUnit 4.3.0 by Sebastian Bergmann.

Usage: phpunit [options] UnitTest [UnitTest.php]
       phpunit [options] <directory>

Code Coverage Options:

  --coverage-clover <file>  Generate code coverage report in Clover XML format.
  --coverage-crap4j <file>  Generate code coverage report in Crap4J XML format.
  --coverage-html <dir>     Generate code coverage report in HTML format.
  --coverage-php <file>     Export PHP_CodeCoverage object to file.
  --coverage-text=<file>    Generate code coverage report in text format.
                            Default: Standard output.
  --coverage-xml <dir>      Generate code coverage report in PHPUnit XML format.

Logging Options:

  --log-junit <file>        Log test execution in JUnit XML format to file.
  --log-tap <file>          Log test execution in TAP format to file.
  --log-json <file>         Log test execution in JSON format.
  --testdox-html <file>     Write agile documentation in HTML format to file.
  --testdox-text <file>     Write agile documentation in Text format to file.

Test Selection Options:

  --filter <pattern>        Filter which tests to run.
  --testsuite <pattern>     Filter which testsuite to run.
  --group ...               Only runs tests from the specified group(s).
  --exclude-group ...       Exclude tests from the specified group(s).
  --list-groups             List available test groups.
  --test-suffix ...         Only search for test in files with specified
                            suffix(es). Default: Test.php,.phpt

Test Execution Options:

  --report-useless-tests    Be strict about tests that do not test anything.
  --strict-coverage         Be strict about unintentionally covered code.
  --disallow-test-output    Be strict about output during tests.
  --enforce-time-limit      Enforce time limit based on test size.
  --disallow-todo-tests     Disallow @todo-annotated tests.
  --strict                  Run tests in strict mode (enables all of the above).

  --process-isolation       Run each test in a separate PHP process.
  --no-globals-backup       Do not backup and restore $GLOBALS for each test.
  --static-backup           Backup and restore static attributes for each test.

  --colors                  Use colors in output.
  --stderr                  Write to STDERR instead of STDOUT.
  --stop-on-error           Stop execution upon first error.
  --stop-on-failure         Stop execution upon first error or failure.
  --stop-on-risky           Stop execution upon first risky test.
  --stop-on-skipped         Stop execution upon first skipped test.
  --stop-on-incomplete      Stop execution upon first incomplete test.
  -v|--verbose              Output more verbose information.
  --debug                   Display debugging information during test execution.

  --loader <loader>         TestSuiteLoader implementation to use.
  --repeat <times>          Runs the test(s) repeatedly.
  --tap                     Report test execution progress in TAP format.
  --testdox                 Report test execution progress in TestDox format.
  --printer <printer>       TestListener implementation to use.

Configuration Options:

  --bootstrap <file>        A "bootstrap" PHP file that is run before the tests.
  -c|--configuration <file> Read configuration from XML file.
  --no-configuration        Ignore default configuration file (phpunit.xml).
  --include-path <path(s)>  Prepend PHP's include_path with given path(s).
  -d key[=value]            Sets a php.ini value.

Miscellaneous Options:

  -h|--help                 Prints this usage information.
  --version                 Prints the version and exits.

  --self-update             Update PHPUnit to the latest version.
server:~ robweaver$ clear

server:~ robweaver$ cd /Volumes/Macintosh\ HD2/Projects/WordPress/
server:WordPress robweaver$ "/usr/bin/php" "/usr/local/bin/phpunit" "--colors" "--log-junit" "/var/folders/wk/kl5ptc6d6gj1bzlw4lr9yhmm0000gn/T/nb-phpunit-log.xml" "/Applications/NetBeans/NetBeans 8.0.1.app/Contents/Resources/NetBeans/php/phpunit/NetBeansSuite.php" "--run=/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests"
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests... To execute these, use --group ajax.
PHPUnit 4.3.0 by Sebastian Bergmann.

Configuration read from /Volumes/Macintosh HD2/Projects/WordPress/phpunit.xml.dist



Time: 3.61 seconds, Memory: 23.50Mb

No tests executed!
server:WordPress robweaver$ phpunit
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests... To execute these, use --group ajax.
PHPUnit 4.3.0 by Sebastian Bergmann.

Configuration read from /Volumes/Macintosh HD2/Projects/WordPress/phpunit.xml.dist

.............................................................   61 / 2669 (  2%)
...........................SS...SSSSSS.............S...SS....  122 / 2669 (  4%)
........SS................S...........SS...SSSSSS............  183 / 2669 (  6%)
.S...SS............SS..........................S.............  244 / 2669 (  9%)
.............................................................  305 / 2669 ( 11%)
...............F.............................................  366 / 2669 ( 13%)
................SSS.......SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..  427 / 2669 ( 15%)
S........................S...................................  488 / 2669 ( 18%)
.S...............................................S.SSSSSSSSS.  549 / 2669 ( 20%)
..................................S..................S.......  610 / 2669 ( 22%)
I............................................................  671 / 2669 ( 25%)
.............................................................  732 / 2669 ( 27%)
.............................................................  793 / 2669 ( 29%)
.............................................................  854 / 2669 ( 31%)
.............................................................  915 / 2669 ( 34%)
............................................................S  976 / 2669 ( 36%)
SSSSSSSSSSSSSS............................................... 1037 / 2669 ( 38%)
............................................................. 1098 / 2669 ( 41%)
............................................................. 1159 / 2669 ( 43%)
............................................................. 1220 / 2669 ( 45%)
............................................................. 1281 / 2669 ( 47%)
....................................S........................ 1342 / 2669 ( 50%)
.........................S................................... 1403 / 2669 ( 52%)
................................................F.F.......... 1464 / 2669 ( 54%)
................E..........SSSSSSSSSSSS...................... 1525 / 2669 ( 57%)
........SSSSSS..............S.......S........................ 1586 / 2669 ( 59%)
...........................................................S. 1647 / 2669 ( 61%)
..................................................S.......... 1708 / 2669 ( 63%)
......SSS......................S....SSSS.....SS.............. 1769 / 2669 ( 66%)
..........S.................................................. 1830 / 2669 ( 68%)
...............S...S...SS.................................... 1891 / 2669 ( 70%)
............................................................. 1952 / 2669 ( 73%)
............................................................. 2013 / 2669 ( 75%)
.....S..........S............................................ 2074 / 2669 ( 77%)
............................S.........................SS..... 2135 / 2669 ( 79%)
............................................................. 2196 / 2669 ( 82%)
............S................................................ 2257 / 2669 ( 84%)
............................................SSS.............. 2318 / 2669 ( 86%)
............................................................. 2379 / 2669 ( 89%)
............................................................. 2440 / 2669 ( 91%)
............................................................. 2501 / 2669 ( 93%)
............................................................. 2562 / 2669 ( 95%)
............................................................. 2623 / 2669 ( 98%)
..............SS.......F........FFSSSSSSSSSSS

Time: 2.6 minutes, Memory: 130.25Mb

There was 1 error:

1) Test_Image_Resize_GD::test_resize_png
basename() expects parameter 1 to be string, object given

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/resize.php:28

--

There were 6 failures:

1) Tests_WP_Date_Query::test_build_mysql_datetime_datetime_non_array
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-1970-01-01 00:00:00
+4646-10-06 14:12:55

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/date/query.php:417

2) Tests_Image_Functions::test_wp_save_image_file
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'image/png'
+'image/jpeg'

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/functions.php:144

3) Tests_Image_Functions::test_inferred_mime_types
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'image/png'
+'image/jpeg'

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/functions.php:235

4) Tests_Image_Editor_GD::test_supports_mime_type
Does not support image/png
Failed asserting that false is true.

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:41

5) Tests_Image_Editor_GD::test_image_preserves_alpha_on_resize
Failed asserting that WP_Error Object (...) is not an instance of class "WP_Error".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:467

6) Tests_Image_Editor_GD::test_image_preserves_alpha
Failed asserting that WP_Error Object (...) is not an instance of class "WP_Error".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:490
                                                                                    
FAILURES!                                                                           
Tests: 2667, Assertions: 10494, Failures: 6, Errors: 1, Incomplete: 1, Skipped: 155.
server:WordPress robweaver$ "/usr/bin/php" "/usr/local/bin/phpunit" "--colors" "--log-junit" "/var/folders/wk/kl5ptc6d6gj1bzlw4lr9yhmm0000gn/T/nb-phpunit-log.xml" 
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests... To execute these, use --group ajax.
PHPUnit 4.3.0 by Sebastian Bergmann.

Configuration read from /Volumes/Macintosh HD2/Projects/WordPress/phpunit.xml.dist

.............................................................   61 / 2669 (  2%)
...........................SS...SSSSSS.............S...SS....  122 / 2669 (  4%)
........SS................S...........SS...SSSSSS............  183 / 2669 (  6%)
.S...SS............SS..........................S.............  244 / 2669 (  9%)
.............................................................  305 / 2669 ( 11%)
...............F.............................................  366 / 2669 ( 13%)
................SSS.......SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..  427 / 2669 ( 15%)
S........................S...................................  488 / 2669 ( 18%)
.S...............................................S.SSSSSSSSS.  549 / 2669 ( 20%)
..................................S..................S.......  610 / 2669 ( 22%)
I............................................................  671 / 2669 ( 25%)
.............................................................  732 / 2669 ( 27%)
.............................................................  793 / 2669 ( 29%)
.............................................................  854 / 2669 ( 31%)
.............................................................  915 / 2669 ( 34%)
............................................................S  976 / 2669 ( 36%)
SSSSSSSSSSSSSS............................................... 1037 / 2669 ( 38%)
............................................................. 1098 / 2669 ( 41%)
............................................................. 1159 / 2669 ( 43%)
............................................................. 1220 / 2669 ( 45%)
............................................................. 1281 / 2669 ( 47%)
....................................S........................ 1342 / 2669 ( 50%)
.........................S......................F.F.......... 1403 / 2669 ( 52%)
................................................F.F.......... 1464 / 2669 ( 54%)
................E..........SSSSSSSSSSSS...................... 1525 / 2669 ( 57%)
........SSSSSS..............S.......S........................ 1586 / 2669 ( 59%)
...........................................................S. 1647 / 2669 ( 61%)
..................................................S.......... 1708 / 2669 ( 63%)
......SSS......................S....SSSS.....SS.............. 1769 / 2669 ( 66%)
..........S.................................................. 1830 / 2669 ( 68%)
....F..........S...S...SS.................................... 1891 / 2669 ( 70%)
............................................................. 1952 / 2669 ( 73%)
............................................................. 2013 / 2669 ( 75%)
.....S..........S............................................ 2074 / 2669 ( 77%)
............................S.........................SS..... 2135 / 2669 ( 79%)
............................................................. 2196 / 2669 ( 82%)
............S................................................ 2257 / 2669 ( 84%)
............................................SSS.............. 2318 / 2669 ( 86%)
............................................................. 2379 / 2669 ( 89%)
............................................................. 2440 / 2669 ( 91%)
............................................................. 2501 / 2669 ( 93%)
............................................................. 2562 / 2669 ( 95%)
............................................................. 2623 / 2669 ( 98%)
..............SS.......F........FFSSSSSSSSSSS

Time: 2.79 minutes, Memory: 131.00Mb

There was 1 error:

1) Test_Image_Resize_GD::test_resize_png
basename() expects parameter 1 to be string, object given

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/resize.php:28

--

There were 9 failures:

1) Tests_WP_Date_Query::test_build_mysql_datetime_datetime_non_array
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-1970-01-01 00:00:00
+4835-10-06 14:12:55

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/date/query.php:417

2) Tests_HTTP_Functions::test_head_404
Reply wasn't array.
Failed asserting that WP_Error Object &0000000049f54afb000000010d3ec26b (
    'errors' => Array &0 (
        'http_request_failed' => Array &1 (
            0 => 'Operation timed out after 5003 milliseconds with 0 bytes received'
        )
    )
    'error_data' => Array &0
) is of type "array".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/http/functions.php:31

3) Tests_HTTP_Functions::test_get_redirect
Reply wasn't array.
Failed asserting that false is of type "array".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/http/functions.php:60

4) Tests_Image_Functions::test_wp_save_image_file
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'image/png'
+'image/jpeg'

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/functions.php:144

5) Tests_Image_Functions::test_inferred_mime_types
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'image/png'
+'image/jpeg'

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/functions.php:235

6) Tests_Post_Query::test_tax_query_single_query_multiple_terms_operator_in
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    0 => 145
-    1 => 146
+    0 => '146'
+    1 => '145'
 )

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/post/query.php:1203

7) Tests_Image_Editor_GD::test_supports_mime_type
Does not support image/png
Failed asserting that false is true.

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:41

8) Tests_Image_Editor_GD::test_image_preserves_alpha_on_resize
Failed asserting that WP_Error Object (...) is not an instance of class "WP_Error".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:467

9) Tests_Image_Editor_GD::test_image_preserves_alpha
Failed asserting that WP_Error Object (...) is not an instance of class "WP_Error".

/Volumes/Macintosh HD2/Projects/WordPress/tests/phpunit/tests/image/editor_gd.php:490
                                                                                    
FAILURES!                                                                           
Tests: 2667, Assertions: 10488, Failures: 9, Errors: 1, Incomplete: 1, Skipped: 155.
server:WordPress robweaver$

Comment 1 Tomas Mysik 2015-04-02 09:50:12 UTC

I will test it but what comes to my mind right now - working directory of NetBeans is the TESTS folder which is usually different from project folder. This means that the phpunit.xml.dist is not used by phpunit CLI. So, the question is what should be the default working directory - project directory or test directory? I don't know and I am not sure whether there is just one correct answer for it :)

However, if one select the phpunit.xml.dist file as a PHPUnit configuration file in Project Properties dialog, I guess it should work then. Will try it.

Thanks for reporting.

Comment 2 Tomas Mysik 2015-04-02 09:54:13 UTC

OK, will not test it since apparently it needs WP installation and proper configuration :/ But I guess that my previous comment is correct - could you please test it? Please, let us know (simply reopen this issue).

Thanks.

Comment 3 Tomas Mysik 2015-04-02 09:55:18 UTC

One more idea - what will happen if you run the NetBeans command from the tests directory (instead of project directory)?

Thanks.

Comment 4 Tomas Mysik 2015-04-02 10:02:03 UTC

Created attachment 152996 [details]
Dialog for PHPUnit configuration

Comment 5 rweaver 2015-04-02 13:13:48 UTC

(In reply to Tomas Mysik from comment #3)
> One more idea - what will happen if you run the NetBeans command from the
> tests directory (instead of project directory)?
> 
> Thanks.

Same results as running from NetBeans:
server:tests robweaver$ "/usr/local/bin/php" "/usr/local/bin/phpunit" "--colors" "--log-junit" "/var/folders/wk/kl5ptc6d6gj1bzlw4lr9yhmm0000gn/T/nb-phpunit-log.xml" "--configuration" "/Volumes/ServerHD2/Projects/WordPress/phpunit.xml.dist" "/Applications/NetBeans/NetBeans 8.0.1.app/Contents/Resources/NetBeans/php/phpunit/NetBeansSuite.php" "--run=/Volumes/ServerHD2/Projects/WordPress/tests"
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHP Notice:  Constant WP_DEBUG already defined in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 20

Notice: Constant WP_DEBUG already defined in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 20
PHP Warning:  require_once(/Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-testlib/utils.php): failed to open stream: No such file or directory in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 34

Warning: require_once(/Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-testlib/utils.php): failed to open stream: No such file or directory in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 34
PHP Fatal error:  require_once(): Failed opening required '/Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-testlib/utils.php' (include_path='.:/usr/local/Cellar/php56/5.6.4/lib/php') in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 34

Fatal error: require_once(): Failed opening required '/Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-testlib/utils.php' (include_path='.:/usr/local/Cellar/php56/5.6.4/lib/php') in /Volumes/ServerHD2/Projects/WordPress/tests/phpunit/wp-mail-real-test.php on line 34

I can run phpunit from the root project folder and get the expected test results.

Comment 6 Tomas Mysik 2015-04-03 05:28:48 UTC

In such case, I guess that the problem could be with our PHPUnit Suite (NetBeansSuite.php), I will try to look at it. Thanks.

Comment 7 Tomas Mysik 2016-03-03 11:54:18 UTC

I just added an option to test project using just 'phpunit' command. See Project Properties dialog, PHPUnit category. Note: Do not forget to select proper XML configuration in this dialog otherwise it will not work.

So,this issue should be fixed now, I hope. Please, let us know if not.

Thanks.

http://hg.netbeans.org/web-main/rev/36719c55ddd8