This also includes the special case whether the java test files (including test suites) should have such redundant actions enabled. This was part of the JUnit TS but I don't see the point of such actions.

See also http://netbeans.org/bugzilla/show_bug.cgi?id=213783

Between comment #0 and comment #1 I do not think I follow what the actual issue is. Can you please give a concrete, complete example of what you observe the behavior to be, and compare to a test spec (give URL)?

>In Ant based Java SE project it is possible
No it's not possible. If you run the class without main the Dialog notifies you that the class has no main.
Doing the test for main or applet in isEnabled is not acceptable from performance reasons. It is called synchronously by EDT and needs to do IO which may be extremely expensive on Windows (the NTFS getBooleanAttribute issue). In fit it was done before and we needed to remove it in NB 7.0 due to EDT slownesses.

(In reply to comment #4)
> >In Ant based Java SE project it is possible
> No it's not possible. If you run the class without main the Dialog notifies you
> that the class has no main.
> Doing the test for main or applet in isEnabled is not acceptable from
> performance reasons. It is called synchronously by EDT and needs to do IO which
> may be extremely expensive on Windows (the NTFS getBooleanAttribute issue). In
> fit it was done before and we needed to remove it in NB 7.0 due to EDT
> slownesses.

In that case it should be removed from Maven based projects as well, right?

(In reply to comment #3)
> Between comment #0 and comment #1 I do not think I follow what the actual issue
> is. Can you please give a concrete, complete example of what you observe the
> behavior to be, and compare to a test spec (give URL)?

Sorry for not be clear enough. The test spec is here: http://wiki.netbeans.org/TS_72_JUnit#Test_suite:_Running_and_Debugging, see the following points:
- "Run single test file"
- "Run single test file II"
- "Run single test file III"

The point is that the spec is saying that if you will execute Run or Test action on the java test file (and Debug and Profile in a similar way), it will execute the same action. This works for And based projects but not for Maven.

The point here is that we either need to remove the part from TS, otherwise Maven support does not behave according to the TS, or update the Maven projects to do so.

IMHO "Run single test file II" from the test spec, and the behavior of j2seproject, is wrong, whereas the behavior of maven is right - S-F6 should always mean to run main(String...), C-F6 should always mean to run JUnit/TestNG. But the policy has been changed in historical memory and I have no idea what the most current UI spec says, if there even is one. (Test specs are not authoritative.)

Can confirm that the current java.j2seproject behavior (and test spec) is wrong. ea79749c1951 supposedly fixed, but perhaps it regressed since then. Bug #119922 comment #9 gives the desired UI, consistent with what I said here in comment #7.

As far as I remember the ea79749c1951 change set just added the possibility to run a test with main class but did not change what is done by "Test File"
The behavior described in bug #119922 comment #9 seems OK to me however it may be interpreted as a regression. "Run File" on test executed the test which seems natural and I am using it everyday.
Reassigning to Petr to determine the correct behavior.

So we need to consider how to handle execution actions (run, debug, profile) meaning

a) run file
b) debug file
c) profile file

and test actions (also run, debug, and profile) meaning

d) run tests
e) debug tests
f) profile tests

when invoked on each one of the following cases: 

A) java source files with main() without existing test file
B) java source files without main() without existing test file
C) java source files with main() with existing test file
D) java source files without main() with existing test file
E) test files with main() 
F) test files without main()

As far as I understand exactly the following combinations should be permitted
while all others should not:

Aa-c
Ca-f
Dd-f
Ea-f
Fd-f

while it is understood that in cases A,C,D all test actions are
performed not in the source file itself but in the accompanying test file,
while in cases E,F all test actions are performed inside the actual file.

-----------------------------------------------------------
From UEX point of view the ideal state would be: all action titles in pop-up menu are enabled when they are applicable and disabled when they are not; action names are self-explanatory so that it is exactly (unambiguously) clear to users what they do; the meaning of equally named actions across various contexts does not change so that users do not need to think about context-dependent concerns. The question whether to display an action disabled or not at all is a little trickier, but it is generally better not to change/shift the ordering of actions within menus depending on context as it would again increase the need for users to concentrate on the UI instead on what they actually need to be done. Naturally, the above stated concerns should be treated equally for Ant and Maven based projects.

However, this all is (as so often) also about the trade-off between purity and utility.

One particual point of concern is performance. Ideally actions should be enabled in menu only if they are applicable in the given context; however, as stated in Comment #4, this should not be achieved at the cost of notable performance degradation, which would be the harder usability "sin". Thus, in cases when isEnabled can not be efficiently evaluated, the subsequent cancellation of the invoked action (with sufficient explanation) is an acceptable compromise.

Another particular point is Run File vs. Test File functionality. AFAIK, in case of Java source files Run File is naturally understood as "run the main() method in this file"; Test File is understood as "run all tests that are defined for this source file". But the same actions are also displayed for test files - here their meaning is not that clear. Run File on test file by definition can be expected to "run main() in the test file", but intuitively can be understood by many as "this is a test file so run the tests". Test File on test file by pure definition should be expected to "run tests that are defined for! (but not in) this test file" what clearly is not very practical, instead Test File on a test file is understood simply as "run all tests that are defined in this test file". Such discrepancies should be avoided whenever possible, however, not necessarily at the cost of utility in cases where something feels intuitive enough for most of users (cf. Comment #9).

-----------------------------------------------------------
In ideal world the actions in the menu would be much more self-explanatory than now. In case of java source right-click menu such hypothetical names could be:

1 Run File invoking main()	S-F6
2 Debug File invoking main()   C-S-F5
3 Profile File invoking main()
4 Run File Tests  C-F6
5 Debug File Tests   C-S-F6
6 Profile File Tests

and in case of test files as implemented now the names could be:

1 Run File invoking main() or Run Tests if no main() present	S-F6
2 Debug File invoking main() or Run Tests if no main() present   C-S-F5
3 Profile File invoking main() or Run Tests if no main() present
4 Run Tests in File  C-F6
5 Debug Tests in File   C-S-F6
6 Profile Tests in File

But such menus of course look awful, take endless time to read, consume much screen space and consequently can hardly be considered for serious use.

===========================================================
To come up with something clear enough but less burdensome to read, I am considering the following. What if we renamed the actions in the pop-up menu simply to:

1 Run File       S-F6
2 Debug File     C-S-F5
3 Profile File
4 Run Tests      C-F6
5 Debug Tests    C-S-F6
6 Profile Tests

and displayed these both for Java source files and for test files ? The only difference would be in case of applying Run File (Debug File, Profile File) on a test file - in such case an attempt would be made to run main() method as is expected by these actions, but, should there be no main() in the test file, the user would be informed about that and queried (e.g. through message box) whether to run tests instead. (Of course in case of applying Run File on standard Java source there would be no such choice.) The seeming downside is the missing word File in connection with running tests, but that IMHO is not that big of a problem, the more so because the user knows that she has invoked the pop-up menu on a file.

Such redefinition would remove most of the ambiguities while not penalizing too much those users who are accustomed to running tests through Run File.
(the change would need to be reflected in other menus as well, like in the editor pop-up menu that appears on test method right-click, as well as in TS)

Opinions ? Comments ?

(In reply to comment #10)
> Fd-f

This is the controversial part, I guess.

> action names are self-explanatory

Which "Run File" evidently is not. "Run Main Method" might work better for Java sources, but this would render it unusable for other languages and miscellaneous file types.

> Test File
> on test file by pure definition should be expected to "run tests that are
> defined for! (but not in) this test file" what clearly is not very practical

Actually it happens, but it is definitely a corner case. ($prj/test/pkg/MyTestUtils.java with static utility methods, $prj/test/pkg/MyTestUtilsTest.java testing that they do what they claim, $prj/test/pkg/RegularTest.java calling MyTestUtils.)

> Run File invoking main() or Run Tests if no main() present

Setting aside the label (obviously a strawman!), the semantics seems confused here: the action would suddenly switch to doing something different if you added or removed a main-method.

> should there be no main() in the test file, the
> user would be informed about that and queried (e.g. through message box)
> whether to run tests instead.

Or just show a message in the status bar or dialog box suggesting they use Run Tests instead.

(You call leaving an action enabled without an appropriate context a "compromise", but in fact you can offer the user much more useful information than you could by just disabling it. For $prj/src/pkg/YourClass.java with no associated test, Run Tests could explain that $prj/test/pkg/YourClassTest.java would have been run had it existed, and can point the user to the File > New template or Tools menu item to create that test suite.)

> the missing word File in connection with running tests

Careful about confusion with Test Project, which also "runs tests", though for a Java project it runs all test suites rather than one test suite (with 1+ test cases).

BTW a minor point which is probably irrelevant here, but just in case: while Run File is always enabled just on single file selections (at least for Java main() methods), Test File (again for Java: JUnit/TestNG, j2seproject, maven) is generally enabled also on multiselections, and even on package nodes where it means "all $dir/*Test.java".

Jesse, thanks for your insight. The complexity here seems even a bit higher than I had though (although I am aware of stuff like project-wide test invocation of course)

In view of the many problems you point out, I try to refocus on two particular points only.

1) the original concern of this bugreport is the apparent inconsistency between Ant and Maven execution actions. It is reported that the same actions have different enabled/disabled state in Ant and Maven context. According to Comment #4 there is strong performance reason to not disable Ant actions in menu even if they are not applicable. In Maven this seems not to be the case ? Nevertheless, the real problem would be if the same action lead to a different final outcome in Ant in Maven, but I do not see that big of a problem if the impossibility to perform action is communicated to users differently in Ant (action enabled, then warning and cancellation) and Maven (action disabled). It would be better if these were the same, but this should not be achieved at the cost of negatively affecting performance (in Ant case) or introducing one unnecessary step for user to discover something can not be done (in Maven case).

Anyway, I like Jesse's idea about leaving all test actions enabled and either executing the respective tests if they exists, or suggesting to user to create tests if they do not exist. Remark: informing about the state of action in status bar is not too good as it can easily be overlooked; status bar seems too visually disconnected from what is going on in pop-up menus. A dialog that pops up as result of invoking an action seems the safer option here.

2) the possibly confusing meaning of Run File vs. Test File:

- first, can we agree that Run File should primarily do the same thing on a test file as it would do on a java source, and only if that proves not possible, it would offer to run tests ? 

- regarding the corner case of invoking tests in java source instead of its test file - is it possible to detect the situation that tests exist both inside java source and in a dedicated test file and to give user a choice ?

- regarding the action name Test File, it seems to me not optimal for this reason: if test files can be run either through main method (or whatever is done by default in Run File) or instead by executing the tests in it, then it would be good to suggest that this action actually does by default something different than Run File, i.e., even with test files that enable Run File it would mean the other option. From this comes my considering the title Run Tests instead of Test File. However, if confusion with project-wide testing is to be definitely avoided, would the name Run File Tests seem more acceptable ? Remark: I do not consider this that much important so if there is doubt, just drop it.

What about traditional NetBeans module projects? They are using Ant for builds and they does no allow you to Run, Debug or Profile files without main method.

This old bug may not be relevant anymore. If you can still reproduce it in 8.2 development builds please reopen this issue.

Thanks for your cooperation,
NetBeans IDE 8.2 Release Boss