Unfortunately I'm not able to reproduce it with NetBeans 6.8 M2,
on Ubuntu 9.04 : On step 6 test fails as expected. 

Could you please try with clean user dir and see does it make any difference ? 

OK.... false alarm.... I could swear that I had tried this with a clean user dir.

I downloaded both M2 AND the latest Dev-200910100201 builds and verified that (with a CLEAN user dir) this is NOT a bug.

Thanks for all your efforts and following up with this bug.

Alexei, I am facing this problem again and again ... I tried the latest dev build (20091015) and it's still reproducible
... re-using the old userdir.

Thanks for the extra info. From the description (re-using old user dir) it looks like the problem could be with stale
compiled data in the userdir. We did not change anything in Junit in this area, might be some changes in Java source
caused this ? 
 Is this problem reproducible with code execution ? E.g please try to create simple java file, execute, change
something, execute and see if the behavior changed accordingly.   

Ok, I've found reproducible test case :
- start IDE with new userdir
- create new J2SE project
- disable Compile On Save
- go to the Test Packages and generate JUnit test (I've chosen 4.x)
- write test method :
   @Test
    public void testHello() {
        assertTrue(HelloWorld.getString().equals("Hello"));
    }
- create HelloWorld class in the same package
public class HelloWorld {
    public static String getString(){
        return "Hello";
    }
}
- switch to Test file and run the test -> it passed
- switch to Hello World and replace "Hello" by null
- save the HelloWorld.java and switch back to test file
- run Test File (Ctrl+F6)
- test passed - and that's wrong !!! .. the problem is that HelloWorld wasn't compiled

Hmm, looks like the same problem is reproducible in src ... so this is not tight to tests :
shortly ...

Run File / Test File compiles only the file user invokes this action on, no other classes even if the class is called
from the one Run/Test-ed are compiled.

Marian, I've simplified your test case w/o Junit(below). Could you please re-assign to appropriate Java category ?

- start IDE with new userdir
- create new J2SE project
- disable Compile On Save
- create HelloWorld class in the same package
public class HelloWorld {
    public static String getString(){
        return "Hello";
    }
}
- Switch to Main class and insert following line into main method:
        System.out.println(new HelloWorld().getString());

- Execute Main class by Shift+F6
 The behavior is correct you will see Hello in the output 

- switch to HelloWorld and replace "Hello" by "Hello Modified"
- save the HelloWorld.java and switch back to Main file
- run Main File (Shift+F6)
- you will yet see "Hello" .. the problem is that HelloWorld wasn't compiled

Alexei is right ... I've done the same with the same result ... so Java - feel free to reassign 

Is this related to Compile on Save? (Are you able to reproduce it with COS disabled?)

> Is this related to Compile on Save? 
I think so ... this is not reproducible with CoS enabled

> Are you able to reproduce it with COS disabled?
Yes, only if CoS is disabled - at least if the file is located in "src" folder ... not sure about files under "test" folder

The same for me - w/o disabling CoS I'm unable to reproduce.

If this worked in 6.7.1 you might consider using Pavel Flaska's favorite 'hg bisect' command. But that would be really
last resort and it will take approx 12 hours to find the changeset that regressed it (not talking about the time to
write a test that has to reliably return true/false). When thinking about it having a test run by hudson to guard
against regression in this would be good to have anyway.

>> Is this related to Compile on Save? 
>I think so ... this is not reproducible with CoS enabled
I am not able to reproduce it in jet-main (both with COS enabled|disabled). From the description it seems that it s caused by regression in FileObjectCrawler 
(Parsing API) which is already fixed in the jet-main (the reason why I am not able to reproduce it).
The strange is that the problem should happen only in case when the compile on save is turned on not off as happened to Marian.
Marian, didn't you exchange COS disabled with COS enabled?

Hm, I guess there may be two intermixed usecases:
-the first one in the issue is probably the problem Tomas was talking about (issue #173876, BTW).
-the second one is the Marian's case from desc6 and av's from desc8 (i.e. run/test single do not compile other modified
source files) - I can reproduce (with disabled CoS), and it seems to work this way since at least NB6.0 (Track Java
Dependencies does not affect this). Note sure if this is a bug at all (definitely not a P1, IMO). BTW: Jesse was
changing something similar for Compile Single in issue #115918.

I would contest that this IS a P1 bug because this was NOT an issue with NetBeans 6.7.1 but is something that was
introduced in the 6.8 codebase 

AND

This is a very common use-case (Someone updates to latest version of NetBeans from an older version.... and a user
performs a CTRL+F6 to run JUnit tests for debugging JUnit tests)

AND

Most people I know that use NetBeans DO NOT use cos (probably because they use Maven as the main project type)

Depends about which of the two (independent) usecases are you talking about:
-the first one cannot be reproduced on a newer build (right? - desc3 states it is no longer reproducible for you in
200910100201) and is most likely a duplicate of issue #173876 (which, BTW, is a P1 bug). As we have another bug for this
(#173876), it does not seem useful to keep this usecase here. (BTW: Maven projects use CoS by default to perform tests,
at least in current dev builds).
-the second one behaves in the very same way in NB6.8dev, NB6.5 and NB6.0 - this is not a P1 IMO (if bug at all).

Tomas Zezula: I know that you were talking to Marial Mirilovic. But I saw the same issue as reported by jcampbell and I
could reproduce the bug only with Compile on Save ENABLED. When COS was disabled, everything seemed to work as expected.
I will try the latest build.

> Marian, didn't you exchange COS disabled with COS enabled?
no for sure it's reproducible while CoS  is disabled

Honza/Tomas, reproducible in NB 6.7.1 ... so do not think this is P1, but I do not think it's P3 either ... it's rather
"not expected" and can cause a lot off misunderstanding and false errors to have sources not compiled before Run/Test
File ... 

As Honza described above the issue are two problems.
1st) reported by jcampbell and verified by Marian Petras is the regression in FileObjectCrawler, duplicate of issue #173876 (already fixed).
2nd) reported by Marian Mirilovic - recompile of dependent files when CoS is turned off in case of j2se projects (does not affect maven).

The fix will make the  run single slow and will require the whole project to be compilable. This will probably make  the run file feature unusable at all. If it is 
what you want  I will integrate it tomorrow.

Tomas, 
> The fix will make the  run single slow and will require the whole project to be compilable. This will probably make  

I am not asking to build the whole project, just classes "the file" depends on ... if we won't do that -> user runs test
that may not run as a part of the "all Tests" testsuite in related project without any changes. 
I understand that users don't want to build the whole project, neither all tests while running "Run Single/Test Single",
but we need to check at least the dependency chain.

>I am not asking to build the whole project, just classes "the file" depends on
Which is the same in src/test case.

... so what about compiling at least the same source root as the file comes from ?

It is definitely intentional that we do not attempt to compile all of src/**/*.java just because you asked to run a unit
test. Doing so would be a major regression for several reasons; see issue #116230. If you have disabled CoS, and you
have made changes to src, and wish to use build/classes for unit tests, you need to either run a full project build or
compile the specific files you have modified. (You can also Test Project which first builds everything.)

As has been pointed out, as of issue #115918 we already compile dependent classes in the same source root. I do not see
any way in general to compile dependent classes in another source root, such as src/p1/Utils.java when
test/p2/SomeTest.java is being run (and SomeTest somehow uses Utils); javac only permits one -d argument regardless of
the size of the sourcepath, so its implicit compilation feature cannot be used.

For the limited case that a "matching" source file is out of date, e.g. src/p/C.java when test/p/CTest.java is being
run, we could try to recompile C.java to build/classes before proceeding with the compilation of CTest.java. I would be
very careful about making any changes of this sort, though; BuildImplTest.java must include a new test case for it.

The original case (COS) was already fixed.
The Marian's case (COS disabled):
We can only allow recompile inside single root (the same root as the executed file is from) as Jesse explained above.
The fix has to remove the -do-not-recompile target from run (test) single, introduced by fix of issue #116230. The  removal may create some regression as 
nobody knows the reason why it was there. I will try and we will see.

Fixed in jet-main b9562b825575

I got the same problem with my NetBeans 8.0.2
IDE does not compile modified unit tests because of it mistakenly believes that source file was not changed ever if it is true. Forced run of "Scan for External Change" feature temporary resolves issue.