This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 207967 - Testng test marked as success while it actually failed
Summary: Testng test marked as success while it actually failed
Status: RESOLVED INVALID
Alias: None
Product: java
Classification: Unclassified
Component: TestNG (show other bugs)
Version: 7.1
Hardware: PC Linux
: P3 normal (vote)
Assignee: Lukas Jungmann
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-01 02:07 UTC by tanyehzheng
Modified: 2012-02-06 02:01 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
test1 is erroneously marked as passed (7.02 KB, image/png)
2012-02-06 02:01 UTC, tanyehzheng
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tanyehzheng 2012-02-01 02:07:17 UTC
I have test code as below. 
test1 passes the test and test2 failed when I expect both to fail.
But when I run the test in command line, both tests failed.
I'm using netbeans 7.1

@Test(dataProvider="prov")
public void test1(int x, int y){
    System.out.println("x=" + x + ", y=" + y);
    assertEquals(x + y, 3);
}

@Test(dataProvider="prov")
public void test2(int x, int y){
    System.out.println("x=" + x + ", y=" + y);
    assertEquals(x + y, 7);
}

@DataProvider
public Object[][] prov(){
    return new Object[][]{
        {1,2},
        {3,4}
    };
}
Comment 1 Lukas Jungmann 2012-02-06 00:37:08 UTC
actually there are 4 tests:

test1(1,2)
test1(3,4)
test2(1,2)
test2(3,4)

where test1(3,4) and test2(1,2) will fail and test1(1,2) and test2(3,4) will pass, so IDE should report 2 passes and 2 failures which is consistent with what TestNG itself reports:

===============================================
Ant suite
Total tests run: 4, Failures: 2, Skips: 0
===============================================

=> invalid
Comment 2 tanyehzheng 2012-02-06 02:01:08 UTC
Created attachment 115481 [details]
test1 is erroneously marked as passed