Bug 47888 - JUnit Sampler re-uses test object
JUnit Sampler re-uses test object
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: Main
unspecified
All All
: P2 normal (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2009-09-22 17:59 UTC by Sebb
Modified: 2011-09-21 12:28 UTC (History)
1 user (show)



Attachments
Fix to the issue (6.47 KB, patch)
2011-09-19 19:58 UTC, Philippe Mouawad
Details | Diff
Test Plan (5.75 KB, application/xml)
2011-09-19 19:58 UTC, Philippe Mouawad
Details
Test JAR (2.53 KB, application/octet-stream)
2011-09-19 20:00 UTC, Philippe Mouawad
Details
Fix to the issue (6.91 KB, patch)
2011-09-20 07:04 UTC, Philippe Mouawad
Details | Diff
Fix to the issue (7.82 KB, patch)
2011-09-21 11:20 UTC, Philippe Mouawad
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sebb 2009-09-22 17:59:06 UTC
The JUnit sampler currently re-uses the same test object for each sample.

This is not how JUnit tests work, as the test runner creates a new instance of the test class for each test.

Re-using the instance means that tests such as the following contrived example will fail if a sampler is run more than once in a loop:

    private int i = 123;
    public void testRerun(){
        assertEquals(123,i);
        i++;
    }

The sampler needs to recreate the test instance for each sample.
Comment 1 peter lin 2009-09-22 18:05:39 UTC
A couple of notes on the difference between jmeter's sampler and the standard junit runner. For a stress test, we don't want to create a new instance of the test class. When I originally wrote the junit sampler, it was for stress and load testing. You can use it for unit testing, but it would require changing how your unit test is written.

Cedric B of TestNG used to mention this design choice of JUnit as one of the flaws. I got around the issue for stress testing by writing our own test runner.

peter
Comment 2 Sebb 2009-09-22 18:46:05 UTC
I see - good point.

It does mean that there are restrictions on the tests that can be run using JMeter, but that's probably worth it - so it will be documented rather than changed.
Comment 3 peter lin 2009-09-22 18:55:37 UTC
A bit more information on how we used the junit sampler.

Our test cases were mainly for IBM Ascential's MetaData Server. The goal was to load the database with x records using a predefined object graph and ETL data in XML format.

The setup would load that data using our persistence API and then start to run the stress test. One of the stress tests measured performance degradation as the database size increases, so we needed to keep state.

If the sampler creates a new instance of the class, we loose state. To get around that, the custom runner was needed. A compromise would be to add a checkbox to create a new instance with each call. This way, the user has the option of using the same instance or creating a new instance.

hope that info is useful.

peter
Comment 4 Philippe Mouawad 2011-09-19 19:58:16 UTC
Created attachment 27536 [details]
Fix to the issue

Hello,
I implemented the solution described by Peter Lin:
- A checkbox
- Recreate new instance at start of each loop

I will attach the JAR that contains a JUnit 3 and JUnit test cases.

Regards
Philippe Mouawad
Comment 5 Philippe Mouawad 2011-09-19 19:58:54 UTC
Created attachment 27537 [details]
Test Plan

The Test plan
Comment 6 Philippe Mouawad 2011-09-19 20:00:04 UTC
Created attachment 27538 [details]
Test JAR

Class filter:
com.ubikingenierie.jmeter.bug47888
Comment 7 Philippe Mouawad 2011-09-20 07:04:02 UTC
Created attachment 27540 [details]
Fix to the issue

Hello,
I had missed in my previous fix the initialization of :
- testCase in JUnit 3
- protectable in both


Regards
Philippe
Comment 8 Sebb 2011-09-21 01:59:52 UTC
Thanks for the patch; looks OK, except the setProperty/getProperty method calls should provide the default value false.

And ideally there should be a patch for component_reference as well.

However I'm not sure you have implemented quite what was intended, which was to optionally recreate the test object for each sample, not each iteration.

I'm not sure if there is a use case for resetting every iteration.

However, if the option is taken to mean "create new test object for this sample", one could apply that to the first sample (only) in a loop and achieve the same result.
Comment 9 Philippe Mouawad 2011-09-21 11:20:05 UTC
Created attachment 27551 [details]
Fix to the issue

Hello Sebb,
Thanks for your review.
You are right I didn't implement exactly what was described.
So I changed to match the issue description and took into account your 2 others notes.

Regards
Philippe Mouawad
Comment 10 Sebb 2011-09-21 12:09:27 UTC
Thanks, much better.

I made some minor changes, and updated the screenshot:

URL: http://svn.apache.org/viewvc?rev=1173581&view=rev
Log:
Bug 47888 - JUnit Sampler re-uses test object

Modified:
   jakarta/jmeter/trunk/docs/images/screenshots/junit_sampler.png
   jakarta/jmeter/trunk/src/core/org/apache/jmeter/resources/messages.properties
   jakarta/jmeter/trunk/src/core/org/apache/jmeter/resources/messages_fr.properties
   jakarta/jmeter/trunk/src/junit/org/apache/jmeter/protocol/java/control/gui/JUnitTestSamplerGui.java
   jakarta/jmeter/trunk/src/junit/org/apache/jmeter/protocol/java/sampler/JUnitSampler.java
   jakarta/jmeter/trunk/xdocs/changes.xml
   jakarta/jmeter/trunk/xdocs/images/screenshots/junit_sampler.png
   jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml

I'm not sure if the word "sample" is a suitable French translation of the English "sample"; elsewhere the property file seems to use échantillon - would that be better here?
Comment 11 Philippe Mouawad 2011-09-21 12:22:13 UTC
Hi,
Shame on me, I am french and I use english words :-)
Echantillon is the right word.

Regards
Philippe