Bug 38824 - Response Assertion "Ignore Status" cancels preceding assertion failures.
Response Assertion "Ignore Status" cancels preceding assertion failures.
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: HTTP
2.1.1
PC Windows XP
: P5 trivial (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2006-03-02 01:49 UTC by deane.sloan
Modified: 2006-04-30 09:59 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description deane.sloan 2006-03-02 01:49:00 UTC
Overview Description: 
The Response Assertion setting "Ignore Status" causes a Response Assertion to 
cancel/drop any preceding response assertion failures.

Steps to Reproduce:
1) Create a test plan (+thread group etc) with a HTTP Request sampler and two 
child Response Assertions. 
2) Configure the thread group to "Stop Thread" after a sampler error and to 
loop forever.
3) Configure the HTTP Request sampler to a URL where you know it will return a 
certain string successfully.
4) Configure the first child Response Assertion with a pattern set to "NOT" 
contain the string that you know *will be* returned (i.e. it will cause it to 
fail). 
6) Ensure the "Ignore status" checkbox on the first child Response Assertion is 
not set.
5) Configure the last child response assertion with a pattern set to contain 
the string that you know will be returned (i.e. it shouldn't fail). 
6) Tick the "Ignore status" checkbox on the last child Response Assertion.

Actual Results: 
The  HTTP Request sample is not set into an error state when only the first 
Response Assertion is triggered/fails on a pattern. The first assertion is 
correctly recorded in the log as failed - however the HTTP sample shows as 
being in success and the thread continues.

Expected Results: 
When the first Response Assertion is triggered/fails on a pattern, set the HTTP 
Request sample to be in error and stop the thread. The documented behaviour of 
the "Ignore Status" does not indicate that preceding assertions are effectively 
ignored if this value is set.


Build/Platform:
Windows XP SP2
Java 2 Runtime Environment, Standard Edition build 1.5.0_06-b05
Apache JMeter version 2.1.1

Thanks
Comment 1 Sebb 2006-03-02 03:08:15 UTC
This is behaving as designed.

The documentation states:

"When the Ignore Status checkbox is selected, the Response status is forced to
successful before evaluating the Assertion."

The purpose of the checkbox is to allow the response status to be checked, even
if  the request has failed, e.g. it allows one to check for a 404.
Comment 2 deane.sloan 2006-03-02 04:53:51 UTC
Thanks - 

Probably led astray a bit by the doco statement "There is no difference between 
setting up one Assertion with multiple patterns and setting up multiple 
Assertions with one pattern each (assuming the other options are the same). " - 
the statement seems to be incorrect based on your response and observations.
Comment 3 deane.sloan 2006-03-02 09:25:03 UTC
My bad - created a contrived example presuming the issue was a bug rather than 
a doco thing.

Simply put - splitting a check where you want to make your own determination 
about HTTP 400+/500+ responses into multiple response assertions using "Ignore 
Status" won't reliably work - the patterns need to be in the one response 
assertion.

The documentation around response assertion has a statement that would lead you 
to believe otherwise ("There is no difference between..."). 
Just a minor doco thing - but still has the same impact if you made the choice 
to split your assertion up based on the statement...

BTW - this seemed like a viable approach to let a known HTTP 500 error through 
for a period of time by trapping everything but... 

Thanks for taking the time to read this.
Comment 4 Sebb 2006-03-02 12:38:35 UTC
(In reply to comment #2)
> Thanks - 
> Probably led astray a bit by the doco statement "There is no difference 
between 
> setting up one Assertion with multiple patterns and setting up multiple 
> Assertions with one pattern each (assuming the other options are the 
same). " - 
> the statement seems to be incorrect based on your response and observations.

Your example did not use the same options...
Comment 5 Sebb 2006-03-02 12:40:02 UTC
I'll see about improving the documentation for the check-box.
Comment 6 Sebb 2006-04-30 16:59:46 UTC
Documentation has been updated.

Ignore Status should only be used on the first Assertion