This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Product Version = NetBeans IDE Dev (Build 20140731-3ee7dd0e3967) Operating System = Linux version 3.13.0-32-generic running on amd64 Java; VM; Vendor = 1.7.0_60 Runtime = Java HotSpot(TM) 64-Bit Server VM 24.60-b09 org.netbeans.modules.maven.groovy [1.7.1 201403261440] and other Groovy-related modules with the same timestamp. When I have github.com/jenkinsci/jenkins/test/src/test/groovy/hudson/cli/SetBuildParameterCommandTest.groovy open in the editor, Ctrl-F6 claims to launch tests: cd …/jenkinsci/jenkins/test; JAVA_HOME=/space/jdk …/java/maven/bin/mvn -Dtest=hudson.cli.SetBuildParameterCommandTest surefire:test Running NetBeans Compile On Save execution. Phase execution is skipped and output directories of dependency projects (with Compile on Save turned on) will be used instead of their jar artifacts. Yet it fails: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-cli) on project jenkins-test-harness: No tests were executed! (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1] Running tests from the command line works: mvn -Dtest=hudson.cli.SetBuildParameterCommandTest test and after doing that test compilation, then Ctrl-F6 from the IDE will start the test. Yet if you edit the test in the IDE and save, then it stops working again.
Created attachment 154256 [details] Sample project describing the bug There is a class ExampleTest, run the file for the first time: The test works, change the body of the method test to 'assert 2 ==3; ' run again the file and you get the message 'No tests were executed'
This a serious problem: I have a very large maven Java Web project where I want to use Groovy, and I just can't use it because of this bug
Cannot run groovy junit tests from Netbeans using groovy 1.7.10, OpenJDK 1.8 and Netbeans 8.0 and 8.1: "Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-cli) on project testing-ground: No tests were executed! (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]." Tests can be executed form the command line without a problem.
FWIW, I share iluvtr's sentiment: this bug will sting at my company having written hundreds of tests in groovy, rather than java, and test suites improvement campaigns are producing more, as is the case right now.
Created attachment 158358 [details] sample project illustrating test execution issue
Created attachment 158359 [details] Screencast showing how the bug can be observed Screencast to show bug behaviour: - "cleaning" the project works - the first "debug test" command after cleaning works - all subsequent "debug test" commands fail ("no tests found") - cleaning the project makes test debugging work again for 1 execution (which represents a frustrating, but usable workaround) On another project which uses a combination of java and groovy unit tests, I haven't observed this behaviour: the Groovy tests can be debugged repeatedly without issues.
A few of additional relevant observations: when triggering "debug test" for the first time (after a "clean" command), the goal "test-compile surefire:test" is executed. However, on subsequent runs, the "test-compile" goal is omitted and only "surefire:test" remains. This looks like a reasonable optimisation, but running "test-compile surefire:test" leaves the target subdir clean of *.class files, which is what causes subsequent runs to fail. I've looked into it further, compared project files after a command-line "mvn test" run to files left over after an initial, IDE-triggered "debug test" and found the following differences: $ diff /tmp/cli.list /tmp/ide.list 5c5 < drwxrwxr-x 6 tnalfir tnalfir 4096 Feb 3 10:22 target --- > drwxrwxr-x 7 tnalfir tnalfir 4096 Feb 3 10:22 target 32c32,33 < total 16 --- > total 20 > drwxrwxr-x 2 tnalfir tnalfir 4096 Feb 3 10:22 classes 37a39,41 > ./target/classes: > total 0 > 63,64c67,68 < -rw-rw-r-- 1 tnalfir tnalfir 275 Feb 3 10:22 failing.sample.FailingTestCase.txt < -rw-rw-r-- 1 tnalfir tnalfir 4437 Feb 3 10:22 TEST-failing.sample.FailingTestCase.xml --- > -rw-rw-r-- 1 tnalfir tnalfir 274 Feb 3 10:22 failing.sample.FailingTestCase.txt > -rw-rw-r-- 1 tnalfir tnalfir 5173 Feb 3 10:22 TEST-failing.sample.FailingTestCase.xml 75,76c79 < total 8 < -rw-rw-r-- 1 tnalfir tnalfir 6490 Feb 3 10:22 FailingTestCase.class --- > total 0 It is obvious from the output that FailingTestCase.class is missing after the IDE-triggered "debug test" command, which is what is causing the command to fail. Now, what puzzles me is that I have another maven project (mixed Java and Groovy) with a Groovy unit test which runs perfectly each time ("runs"="debug test")...I've aligned the two POMs relatively closely, but for some reason the other project works and this one doesn't. I hope the sample project I've attached will help. IMPORTANT NOTE: I've found what looks like a very practical workaround: I went to Project->Properties->Build->Actions and redefined the "Debug test" Action so that its goal is now "test" instead of "test-compile surefire:test", as was previously the case. This allows me to repeatedly successfully debug the file, albeit a bit slower than it should be because it compiles code every time as opposed to simply running them. It's quite puzzling that before I implemented the workaround, the "test-compile" goal was "magically" removed from the "debug test" action, without any apparent way to block this (apparently broken) optimisation.
(In reply to hefest from comment #7) Hello, thank you for providing further information for the analysis of this bug. When CoS is enabled, and according to the wiki[1], all the maven phases are removed and you're left with the goal. Hence when the action is "mvn test-compile surefire:test", with CoS what will be actually executed is "mvn surefire:test". It seems your observations confirm this. Could you confirm that disabling CoS altogether (in the Project Properties > Build > Compile) creates a reliable environment? I've been also looking at this for #256367 and thinking about disabling CoS when groovy classes are included. This is the current behavior for ant projects. Because you have an environment with a considerable amount of groovy tests I wonder if you could give me an example of a real life performance impact of CoS on your build? i.e. using CoS vs disabling it? I am under the impression that maven already checks whether classes need an update during the compilation and if so the impact of disabling CoS might be negligible. [1]: http://wiki.netbeans.org/FaqCompileOnSave#Using_Compile_on_Save_in_Maven_Projects
(In reply to bruno.flavio from comment #8) Thanks for the immediate response, Bruno. > When CoS is enabled, and according to the wiki[1], all the maven phases are > removed and you're left with the goal. Hence when the action is "mvn > test-compile surefire:test", with CoS what will be actually executed is "mvn > surefire:test". It seems your observations confirm this. > > Could you confirm that disabling CoS altogether (in the Project Properties > > Build > Compile) creates a reliable environment? Indeed: disabling compile-on-save and restoring the "debug test" action to its original "test-compile surefire:test" goals, repeatly causes both goals to be specified, thus resulting in a successful execution. > Because you have an environment with a considerable amount of groovy tests I > wonder if you could give me an example of a real life performance impact of > CoS on your build? i.e. using CoS vs disabling it? I am under the impression > that maven already checks whether classes need an update during the > compilation and if so the impact of disabling CoS might be negligible. I'm not sure what to tell you about performance: I enabled CoS, modified a groovy test, saved it and - contrary to my expectations - could not observe any CPU, disk, or IDE activity. I would have expected a progress indicator to at least flash into and out of existence to communicate the fact that something was triggered and is now complete.
(In reply to hefest from comment #9) > I'm not sure what to tell you about performance: I enabled CoS, modified a > groovy test, saved it and - contrary to my expectations - could not observe > any CPU, disk, or IDE activity. I would have expected a progress indicator > to at least flash into and out of existence to communicate the fact that > something was triggered and is now complete. Sorry, I didn't make myself clear: Is there a noticeable difference in the time it takes to *run* a set of tests with CoS disabled when compared with the CoS enabled (which this bug report shows that only works once after compile)?
(In reply to bruno.flavio from comment #10) > Sorry, I didn't make myself clear: > Is there a noticeable difference in the time it takes to *run* a set of > tests with CoS disabled when compared with the CoS enabled (which this bug > report shows that only works once after compile)? These are long-running integration tests, in which the overwhelming majority of time is spent in actual test execution logic and they take too long for me to run them from the IDE as I can't leave the IDE busy for 2-3h.
Marking as duplicate of #256367: The Compile On Save feature is not handling the compilation of groovy classes in maven projects correctly. *** This bug has been marked as a duplicate of bug 256367 ***