This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 227674 - Support Failsafe Plugin
Summary: Support Failsafe Plugin
Status: REOPENED
Alias: None
Product: projects
Classification: Unclassified
Component: Maven (show other bugs)
Version: 8.1
Hardware: All All
: P3 normal with 9 votes (vote)
Assignee: Tomas Stupka
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-19 18:43 UTC by spraguep
Modified: 2017-02-08 19:29 UTC (History)
0 users

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description spraguep 2013-03-19 18:43:07 UTC
Currently NetBeans supports surefire for runings/debugging tests. I would like to also be able to use the same functionality with failsafe.

Currently only the last 3 of the following 5 actions can be made to work with failsafe. However it is rather tedious to do this on a project by project basis, editing the actions for each one. In addition the 2 most useful ones 'Run Focused Test' and 'Debug Focused Test' cannot be made to work at all.

Run Focused Test
---------------------
mvn -Dtest=com.company.ExampleTest#testMethod test-compile surefire:test

Debug Focused Test
---------------------
mvn -Dtest=com.company.ExampleTest#testMethod -DforkMode=once "-Dmaven.surefire.debug=-Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=52693" -Djpda.listen=true -Djpda.address=52693 test-compile surefire:test

Test Project
---------------------
mvn test

Test File
---------------------
mvn -Dtest=com.company.ExampleTest test-compile surefire:test

Debug Test
---------------------
mvn -Dtest=com.company.ExampleTest -DforkMode=once "-Dmaven.surefire.debug=-Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=52693" -Djpda.listen=true -Djpda.address=52693 test-compile surefire:test


I think the only thing needed to support failsafe is changing 'surefire' to 'failsafe' and using '-Dit.test' instead of '-Dtest'. 

The hard part is figuring out how to pick or toggle between surefire/failsafe or support them both at the same time in a single project.
Comment 1 Milos Kleint 2013-10-03 10:50:04 UTC
isn't the file pattern for failsafe tests FooIT.java ? and for surefire FooTest.java?
Comment 2 antarus 2014-07-31 16:13:33 UTC
Hello,

or, maybe, add the possibility to add in custom maven goals, an IDE Selection Expression $ {methodName} referencing the focus method...
Comment 3 Martin Balin 2016-07-07 08:39:08 UTC
This old bug may not be relevant anymore. If you can still reproduce it in 8.2 development builds please reopen this issue.

Thanks for your cooperation,
NetBeans IDE 8.2 Release Boss
Comment 4 smil 2016-07-23 14:13:50 UTC
I don't see failsafe support anywere in 8.1 or 8.2. 

Please reopen.
Comment 5 smil 2016-09-26 18:12:10 UTC
Surefire and Failsafe has a list of patterns, which identifies the integration tests and the normal tests. 

This information is in the maven project available.

The running is a bit different I guess, because failsafe does not fail in the test goal only in the verify. Additionally it has pre and port integration tests phases in order to be able to start/stop the infrastructure.

"mvn verify" is most likely not a good way of invoking, because it will run TEST phase also. Maybe maven.surefire.skip=true.
Comment 6 TrevorGetty 2017-02-08 19:18:37 UTC
Guys -> This is one of the biggest drawbacks of using netbeans at present.
I can run focused unit tests, easily within your IDE, but when it comes to integration tests, using FAILSAFE, it fails badly.

If you run a full build, of course all the tests will run by default, but you do not get any integration with the test pane, or allow the user to run or debug a focused test method. 

This has been an issue now since 2013, and I have verified that it is still an issue in 8.1 and 8.2.


What we need is the same integration with the Test Results pane which currently exists for surefire to work regardless of whether it is surefire or failsafe.

I can see an related fix, to show the test output results but it clearly states that it doesn't fix the result pane viewing.
Bug 192797 - Linking test failure report for maven-failsafe-plugin
Comment 7 TrevorGetty 2017-02-08 19:29:18 UTC
(In reply to smil from comment #5)
> Surefire and Failsafe has a list of patterns, which identifies the
> integration tests and the normal tests. 
> 
> This information is in the maven project available.
> 
> The running is a bit different I guess, because failsafe does not fail in
> the test goal only in the verify. Additionally it has pre and port
> integration tests phases in order to be able to start/stop the
> infrastructure.
> 
> "mvn verify" is most likely not a good way of invoking, because it will run
> TEST phase also. Maybe maven.surefire.skip=true.


To run a focused test you can use:
To run integration tests "mvn verify" is correct, and it will run all phases before that -> "mvn test" also runs the phases before that by default.

- I think to run the tests you are looking at:
mvn verify

- To just run tests in a package
mvn verify -Dit.test=${packageClassName}

- To just run a single test in a package
mvn verify -Dit.test=${packageClassName}#${methodName} 

To debug you can use the property:
- to debug a single test 
mvn verify -Dit.test=${packageClassName}#${methodName} -Dmaven.failsafe.debug

Of course in the IDE it will need to open the listener etc, I have a custom goal to achieve this, but it still doesn't integrate the results into the test pane, so we need at least 2 enhancements.

1) to be able to run the above commands in the IDE by default outside of a custom goal - this will just output the results to the console.
2) report these correctly into the Test Results pane, or a new Integration Test Results pane, I am not sure -> i'll leave that to your UI experts.