This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 167429 - [68cat] Summary display for "Test project" is wrong
Summary: [68cat] Summary display for "Test project" is wrong
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: JUnit (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker with 4 votes (vote)
Assignee: Theofanis Oikonomou
URL:
Keywords: REGRESSION
: 172750 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-06-21 19:45 UTC by tkellerer
Modified: 2012-10-27 01:35 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Summary display in NB 6.5 (32.21 KB, image/png)
2009-06-21 19:45 UTC, tkellerer
Details
Summary display in NB 6.7 (42.29 KB, image/png)
2009-06-21 19:46 UTC, tkellerer
Details
Single JUnit run which is displayed wrongly (25.27 KB, image/png)
2009-06-21 19:47 UTC, tkellerer
Details
Single JUnit run in NB 6.5 (27.77 KB, image/png)
2009-06-21 19:52 UTC, tkellerer
Details
A sample Project. (12.14 KB, application/octet-stream)
2009-11-02 10:01 UTC, stefan79
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tkellerer 2009-06-21 19:45:30 UTC
When I run "Test project" in NB 6.5 it gives me a summary stating that 405 test cases have been executed. They are all
listed as succesfull

Running the same tests (identical source code) in NB 6.7RC3 only shows 120 test cases and one that is "ABORTED"

The one that is marked as aborted does finish correctly the output window shows the correct feedback from JUnit, but the
NB GUI somehow thinks it has been aborted, and stops counting after that one.

Of course the tests run without problems from within a regular (non-NB) ant script.
Comment 1 tkellerer 2009-06-21 19:45:59 UTC
Created attachment 83855 [details]
Summary display in NB 6.5
Comment 2 tkellerer 2009-06-21 19:46:22 UTC
Created attachment 83856 [details]
Summary display in NB 6.7
Comment 3 tkellerer 2009-06-21 19:47:09 UTC
Created attachment 83857 [details]
Single JUnit run which is displayed wrongly
Comment 4 tkellerer 2009-06-21 19:52:22 UTC
Created attachment 83858 [details]
Single JUnit run in NB 6.5
Comment 5 tkellerer 2009-06-25 21:49:34 UTC
OK, I think I found the reason for this strange behaviour.

One of my tested classes prints information to System.out as it expects a "regular" console it uses a CR to "erase" the
console line, so it's doing a 

System.out.print('\r');

apparently this causes trouble with the JUnit integration in NB 6.7

If I configure the tested class to not print the '\r' (it's not necessary for the test anyway) then everything is fine.

The same tests run without problems on 6.5!
Comment 6 emcmanus 2009-09-09 10:38:05 UTC
I'm seeing the same problem, but unfortunately the workaround reported by tkellerer doesn't work for me. I removed all
prints from the test in question but NetBeans still reports ABORTED after successfully running each test case. I
actually have two tests that have started doing this since yesterday for reasons that escape me. I tried deleting the
var/cache directory but to no avail. There are no messages in the messages.log corresponding to the test run.
Comment 7 emcmanus 2009-09-09 10:49:06 UTC
Turns out the problem *was* due to prints, diagnostic prints in the code under test, apparently. I found a dead simple
workaround, which was just to add System.out.println() to the @After method (the tearDown() that is generated
automatically in the test template). Adding it to only the @AfterClass method was not enough.
Comment 8 emcmanus 2009-09-09 11:25:29 UTC
I isolated the problem to a System.out.print() (not println()) in my code. My suspicion is that if a test prints an
incomplete line to System.out, NetBeans does not see a status line printed by JUnit.
Comment 9 stefan79 2009-10-29 17:07:00 UTC
I´ve the same problem at the latest build (200910281401).
But I haven´t found any System.out.print (I set a breakpoint to System.out ant tested ist).
Comment 10 Victor Vasilyev 2009-10-29 18:43:49 UTC
stefan79,

What is a project you are trying to test (Java Application, Web Project, ...)?
What is a test action you are using ("Test Alt+F6" against the project, "Test File Ctrl+F6" against a class under test,
...)?

"the same problem" - What does it mean in your case? 
Note, the current implementation relies on an XML report generated after JUnit testing, but not on the results from
System.out or System.err. So, I think, your problem is not the same.

Could you please, publish here steps to reproduce the issue with clear description what is wrong. 
Comment 11 stefan79 2009-10-29 19:06:25 UTC
-) Project=Java Application
-) First I tried Test on the project "ALT+F6", then "Test File Strg+F6" and after that "Debug Test File".

> "the same problem" - What does it mean in your case? 
The same problem means, that the test-result is aborted.
The Test is ok, because the same test runs at CruiseControl without problems.

>Note, the current implementation relies on an XML report generated after JUnit testing, 
> but not on the results from System.out or System.err. So, I think, your problem is not the same.
Maybe it´s not the same problem, but I see the same result.

> Could you please, publish here steps to reproduce the issue with clear description what is wrong. 
I can reproduce it on my local machine, but I´m not allowed to give the sources out of house.
Can you put some messages to logfile/junit-window, why the action is aborted?
Comment 12 Victor Vasilyev 2009-10-29 19:53:38 UTC
> I can reproduce it on my local machine, but I´m not allowed to give the sources out of house.
No problems. Not all projects are open-sourced yet :-(

> Can you put some messages to logfile/junit-window, why the action is aborted?
The IDE uses a <junit> Ant task to executing the tests.
You can get more info about the process from the "Output Ctrl+4" window.
Note, you can also increase spam of the Ant via the "Tools/Options/Miscellaneous/Ant/Verbosity Level:". Choose the
"Debug" level to have all possible info.

Another source for info is the JUnit test reports (XML files) in the <JavaApplicationProjectDir>/build/test/results
directory that are generated after action "Test Alt+F6" against the project.

Please, report here the results of your hunt for the bug. 
Comment 13 stefan79 2009-10-29 20:20:57 UTC
It´s not a System.out.print.
At tearDown, I made an System.out.close() (not direct in the class, but 7 classes deeper).

Can you give a better Message like "You idiot have closed the Outputstream". :-)
Comment 14 Victor Vasilyev 2009-10-29 23:18:23 UTC
stefan79,

Cool! Such solutions must be open-sourced to make more money :-)
As far as I understood, your problem is solved?
Comment 15 Victor Vasilyev 2009-10-30 12:09:02 UTC
Since the Issue 171050 has been fixed, the content of the output stream should not impact the test results, because an
XML report is used as a source to generate the JUnit test report in the IDE.

I'll will close this issue as duplicate of the Issue 171050.
Please, re-open this issue if you'll see this issue again in the latest versions of the IDE.

*** This issue has been marked as a duplicate of 171050 ***
Comment 16 stefan79 2009-11-02 10:00:15 UTC
It doesn´t work at Version Build 200910310201.

A close on System.out has the effect, does the test becames ABORTED.
I´ll attach a sample-project.
Comment 17 stefan79 2009-11-02 10:01:02 UTC
Created attachment 90408 [details]
A sample Project.
Comment 18 Jiri Kovalsky 2009-11-03 20:26:06 UTC
Reported by NetCAT 6.8 participant. Sample project attached -> removing INCOMPLETE keyword.
Comment 19 dwette 2010-04-20 18:49:22 UTC
I am also having a problem with the ABORTED issue while running tests in Netbeans 6.8 (Mac OS X 10.6.3, java build 1.6.0_19-b04-265-10M3044). In my case I can turn it on and off. It occurs in some tests where I check that an exception is thrown. E.g.

        try {
            reader.read(url, dataStoreMock);
            fail("Expected DataReaderException");
        } catch (DataReaderException ex) {

        }

The tested code logs (log4j) an error before it throws the exception. It uses the logger error() method that takes an Object and a Throwable. The ABORTED problem occurs the first time the exception is thrown (and logged). The log output is interrupted in the NB IDE in the middle of the stack trace and then ABORTED appears. However, all the tests still run (I get all the xml reports in the build/test directories of my modules). The strange part is that the IDE stops displaying the log message in the middle of the stack trace. If I change the tested code to log using the error method that takes only an Object, the ABORTED problem goes away.

In other words: Logger.error(Object, Throwable) in tested code causes ABORTED when my test runs; logger.error(Object) does not.

This is from the xml output. Note where I comment that IDE display ends.

<system-out><![CDATA[20 Apr 2010 13:33:02,032 ERROR IdtReader Error reading file:/Volumes/Boeing/clcmgr3_trunk/modules/clccore/build/test/unit/data/com/boeing/clcmgr/core/io/idt/BadNumParams.idt, Unexpected end-of-file encountered
com.boeing.clcmgr.data.api.DataReaderException: Unexpected end-of-file encountered
	at com.boeing.clcmgr.core.io.idt.IdtReader.parseNames(IdtReader.java:520)
	at com.boeing.clcmgr.core.io.idt.IdtReader.parseRun(IdtReader.java:289)
	at com.boeing.clcmgr.core.io.idt.IdtReader.read(IdtReader.java:164)
	at com.boeing.clcmgr.core.io.idt.IdtReaderTest.testBadNumParams(IdtReaderTest.java:129)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
// the IDE stops displaying here
// and what's left below is only int he XML
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at junit.framework.TestCase.runTest(TestCase.java:168)
	at org.netbeans.junit.NbTestCase.access$200(NbTestCase.java:88)
	at org.netbeans.junit.NbTestCase$2.doSomething(NbTestCase.java:336)
	at org.netbeans.junit.NbTestCase$1Guard.run(NbTestCase.java:273)
	at org.netbeans.junit.NbTestCase.runBare(NbTestCase.java:355)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at org.netbeans.junit.NbTestCase.run(NbTestCase.java:213)
	at junit.framework.TestSuite.runTest(TestSuite.java:232)
	at junit.framework.TestSuite.run(TestSuite.java:227)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
Comment 20 dwette 2010-07-20 18:23:10 UTC
This is still a problem for me with Netbeans IDE 6.9.
Comment 21 jbeskin 2011-02-04 14:29:26 UTC
I realize this is an old bug, but it's among the first pages I found while looking for the same issue. 

This seems to be a race condition, since it doesn't happen all the time. The race condition appears to be between log4j's exception handling and NetBeans view of stdout.

It is possible to reduce the race condition by adding a System.out.println("") immediately after any logging. Putting it in the @After part of the unit test doesn't work if you log multiple exceptions. Finally, it helps to print something at the beginning of the test as well. 

So if you are having this problem with log4j Throwable logging causing Netbeans to ABORT, use the following pattern which seems to reduce the window for the race condition. (I don't believe it prevents the issue.)

@Test
public void someTest() {
  System.out.println("someTest");
  // Some code that logs a Throwable via log4j
  ... 
  System.out.println("");
}
Comment 22 Yulia Novozhilova 2011-08-09 11:34:31 UTC
*** Bug 172750 has been marked as a duplicate of this bug. ***
Comment 23 Theofanis Oikonomou 2012-10-26 08:10:00 UTC
Fixed: http://hg.netbeans.org/core-main/rev/4739cc2c2e94
Comment 24 Quality Engineering 2012-10-27 01:35:47 UTC
Integrated into 'main-golden', will be available in build *201210270001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/4739cc2c2e94
User: Theofanis Oikonomou <theofanis@netbeans.org>
Log: Issue #167429 - [68cat] Summary display for "Test project" is wrong