Bug 64196

Summary:	Recurse into sub samplers more deeply when checking assertions
Product:	JMeter - Now in Github	Reporter:	Wicky Tam <tanweiji>
Component:	HTTP	Assignee:	JMeter issues mailing list <issues>
Status:	RESOLVED FIXED
Severity:	normal	CC:	p.mouawad
Priority:	P2	Keywords:	FixedInTrunk
Version:	5.2.1
Target Milestone:	JMETER_5.3.0
Hardware:	All
OS:	All
Attachments:	the jmeter script the date captured by wireshark result tree script & screenshot for point 3 Recurse deepr into subsamplers for assertion checks Test plan to show that sub-sub-samples are not considered by assertions Recurse into sampleResults to construct getSampleList

Description Wicky Tam 2020-03-04 14:30:06 UTC

Response Assertion do wrong when there are sub-requests

I use a web server program "webtours" which is a part of loadrunner11.
the Home page of "webtours" contain some embedded resources(*.html & *.pl & *.png),so the http samplers to this page can generate sub-samples.and some sub-samples can also generate sub-sub-samples.

1.I make a HTTP main sampler to visit this web page
2.set the Attribute "Retrieve All Embedded Resources from HTML Files" in this sampler to be "Yes".
3.I select a Response Assertion into this sampler.
4.set the Attribute "Apply to:" of Response Assertion to be "Main sample and sub-samples"
5.I select a View Results Tree into the Thread Group.

I found three things that I think is wrong!
1.I fill a string which is Contained in the response of only one sub-request into the Attribute “Patterns to Test”.I think the sampler will be successful because I can find the string in the web page.But the real result is failure because the string is not found in the responses of main sampler and other sub-samplers. it won't be successful even if the string is found in the response of one sub-sampler.
I think it is wrong! the result should be successful when the string is found in the response of any sampler(including sub_sampler)

2. Response Assertion won't found the pattern strings in the sub-sub-samples.So does the Regular Expression Extractor. The Attribute "Apply to:" is used for Main sample and sub-samples,but does nothing to sub_sub_samples.Sometimes it gets me in trouble.

3. Let's look into the View Results Tree,you will find there are some sub_sampler and a main sampler under the main sampler.Yes, A duplicate main sampler is under the main sampler.it does wrong sometimes! For example, the Attribute "Apply to:" of Response Assertion is set to be "sub-samples only". Fill a string which is Contained in all responses of sub-request(but no the main_sampler) into the Attribute “Patterns to Test”. the real result is failure because the string is not found in the duplicate main sampler. is it right? I don't think so.

Comment 1 Philippe Mouawad 2020-03-07 18:39:52 UTC

Hello 

Could you clarify points 2 and 3 with examples and screenshots showing problems ?

Also it would be better to create 1 issue per item .

Thanks

Comment 2 Wicky Tam 2020-03-08 03:45:00 UTC

Created attachment 37069 [details]
the jmeter script

Comment 3 Wicky Tam 2020-03-08 04:41:04 UTC

Created attachment 37070 [details]
the date captured by wireshark

Comment 4 Wicky Tam 2020-03-08 04:42:41 UTC

Created attachment 37071 [details]
result tree

Comment 5 Wicky Tam 2020-03-08 09:07:01 UTC

Created attachment 37072 [details]
script & screenshot for point 3

based on the script of point 2，but change the regular expressions of Response Assertion to be “(?i)/body”。this  regular expressions can  match in sub_sampler and not in main sampler.

Comment 6 Wicky Tam 2020-03-08 09:09:35 UTC

attachment 37069 [details]/37070/37071 is for point 2.attachment 37072 [details] is for point 3.

Comment 7 Wicky Tam 2020-03-08 09:34:02 UTC

Thanks for your reply. It's the first time that I use this system to submit the bug. I will create 1 issue per item.

(In reply to Philippe Mouawad from comment #1)
> Hello 
> 
> Could you clarify points 2 and 3 with examples and screenshots showing
> problems ?
> 
> Also it would be better to create 1 issue per item .
> 
> Thanks

Comment 8 Felix Schumacher 2020-03-08 14:48:37 UTC

Created attachment 37073 [details]
Recurse deepr into subsamplers for assertion checks

I have not found a webpage, that would generate sub-sub-samplers for me, but I think I know where the recursion has to happen. Would you be able to compile JMeter with the attached patch for yourself?

Comment 9 Felix Schumacher 2020-03-08 15:26:46 UTC

Created attachment 37074 [details]
Test plan to show that sub-sub-samples are not considered by assertions

The attached plan is adapted from the originally submitted plan. It uses a demo site, that is currently online and can be used for testing.

It checks, that there is no "file: test1\d.css" found in any of the samples (including sub-samples). If the patch is applied and JMeter recurses into sub-sub-samples, the test plan fails (as intended).

So one of the two aspects of this bug should be fixed with the patch.

Comment 10 Felix Schumacher 2020-03-08 16:00:34 UTC

Created attachment 37075 [details]
Recurse into sampleResults to construct getSampleList

This will recurse deeper into sub sub samples to get sample lists to be used by regex extractor (for example).

Comment 11 Wicky Tam 2020-03-09 12:06:59 UTC

Due to the failure of patching with the “patch” command, I manually modified the source code of the program according to the patch file you gave me.

and I tried my best to compile JMeter, but I failed.Because I'm not good at programming,so I can't do anything now.
so could u please help me to compile or do anther thing to solve the problem？Thanks！

(In reply to Felix Schumacher from comment #8)
> Created attachment 37073 [details]
> Recurse deepr into subsamplers for assertion checks
> 
> I have not found a webpage, that would generate sub-sub-samplers for me, but
> I think I know where the recursion has to happen. Would you be able to
> compile JMeter with the attached patch for yourself?

Comment 12 Felix Schumacher 2020-03-10 19:46:17 UTC

The fix has been committed. Could you please test the next nightly build?

commit 1a2f24ab25f2a571f7bcb599b4be692eb47dde92
AuthorDate: Sun Mar 8 15:26:25 2020 +0100

    Recurse into sub samplers more deeply when checking assertions
    
    With this change we recurse up to three levels into sub results of SampleResults
    instead of only one level deep.
    
    This helps in case of parallel downloads of embedded elements of a web page, which
    get stored as sub-sub-sampleresults.
    
    Bugzilla Id: 64196
---
 .../org/apache/jmeter/threads/JMeterThread.java    | 38 +++++++++++++---------
 xdocs/changes.xml                                  |  1 +
 2 files changed, 24 insertions(+), 15 deletions(-)

commit 30a71a3112e436889d69ab9d2cb5403f4a3ab59c
AuthorDate: Sun Mar 8 16:58:35 2020 +0100

    Recurse into sampleResults for getSampleList
    
    With this change we collect the sampleList of a SampleReult by recursing up to
    three levels into sub results of SampleResults instead of only one level deep.
    
    This helps in case of parallel downloads of embedded elements of a web page, which
    get stored as sub-sub-sampleresults. Before this change a regex Post Processor would
    not see those sub-sub-sampleresults.
    
    Bugzilla Id: 64196
---
 .../jmeter/testelement/AbstractScopedTestElement.java | 19 ++++++++++++++++++-
 xdocs/changes.xml                                     |  1 +
 2 files changed, 19 insertions(+), 1 deletion(-)

Comment 13 Wicky Tam 2020-03-11 08:36:24 UTC

I downloaded the nightly build Jmeter（LATEST (r1659-30a71a3112e436889d69ab9d2cb5403f4a3ab59c)）and tested it.

I find that regex extractor and  Response Assertion can find the string in sub-sub-sampler in this version of jmeter.

It solve the issue of point 2.

what about other  issues?


(In reply to Felix Schumacher from comment #12)
> The fix has been committed. Could you please test the next nightly build?
> 
> commit 1a2f24ab25f2a571f7bcb599b4be692eb47dde92
> AuthorDate: Sun Mar 8 15:26:25 2020 +0100
> 
>     Recurse into sub samplers more deeply when checking assertions
>     
>     With this change we recurse up to three levels into sub results of
> SampleResults
>     instead of only one level deep.
>     
>     This helps in case of parallel downloads of embedded elements of a web
> page, which
>     get stored as sub-sub-sampleresults.
>     
>     Bugzilla Id: 64196
> ---
>  .../org/apache/jmeter/threads/JMeterThread.java    | 38
> +++++++++++++---------
>  xdocs/changes.xml                                  |  1 +
>  2 files changed, 24 insertions(+), 15 deletions(-)
> 
> commit 30a71a3112e436889d69ab9d2cb5403f4a3ab59c
> AuthorDate: Sun Mar 8 16:58:35 2020 +0100
> 
>     Recurse into sampleResults for getSampleList
>     
>     With this change we collect the sampleList of a SampleReult by recursing
> up to
>     three levels into sub results of SampleResults instead of only one level
> deep.
>     
>     This helps in case of parallel downloads of embedded elements of a web
> page, which
>     get stored as sub-sub-sampleresults. Before this change a regex Post
> Processor would
>     not see those sub-sub-sampleresults.
>     
>     Bugzilla Id: 64196
> ---
>  .../jmeter/testelement/AbstractScopedTestElement.java | 19
> ++++++++++++++++++-
>  xdocs/changes.xml                                     |  1 +
>  2 files changed, 19 insertions(+), 1 deletion(-)

Comment 14 Philippe Mouawad 2020-03-14 18:00:58 UTC

(In reply to Wicky Tam from comment #13)
> I downloaded the nightly build Jmeter（LATEST
> (r1659-30a71a3112e436889d69ab9d2cb5403f4a3ab59c)）and tested it.
> 
> I find that regex extractor and  Response Assertion can find the string in
> sub-sub-sampler in this version of jmeter.
> 
> It solve the issue of point 2.
> 
> what about other  issues?

Maybe a thanks would be a good start .

For point 1, please open a separate enhancement request. AFAIK , it always worked like this , I am not sure we can change behaviour, so we would add a new option, this needs to be discussed

For point 3, please also open a separate bug and clarify how it differs from 1. It's not clear for me.

Thank you in advance

> 
> 
> (In reply to Felix Schumacher from comment #12)
> > The fix has been committed. Could you please test the next nightly build?
> > 
> > commit 1a2f24ab25f2a571f7bcb599b4be692eb47dde92
> > AuthorDate: Sun Mar 8 15:26:25 2020 +0100
> > 
> >     Recurse into sub samplers more deeply when checking assertions
> >     
> >     With this change we recurse up to three levels into sub results of
> > SampleResults
> >     instead of only one level deep.
> >     
> >     This helps in case of parallel downloads of embedded elements of a web
> > page, which
> >     get stored as sub-sub-sampleresults.
> >     
> >     Bugzilla Id: 64196
> > ---
> >  .../org/apache/jmeter/threads/JMeterThread.java    | 38
> > +++++++++++++---------
> >  xdocs/changes.xml                                  |  1 +
> >  2 files changed, 24 insertions(+), 15 deletions(-)
> > 
> > commit 30a71a3112e436889d69ab9d2cb5403f4a3ab59c
> > AuthorDate: Sun Mar 8 16:58:35 2020 +0100
> > 
> >     Recurse into sampleResults for getSampleList
> >     
> >     With this change we collect the sampleList of a SampleReult by recursing
> > up to
> >     three levels into sub results of SampleResults instead of only one level
> > deep.
> >     
> >     This helps in case of parallel downloads of embedded elements of a web
> > page, which
> >     get stored as sub-sub-sampleresults. Before this change a regex Post
> > Processor would
> >     not see those sub-sub-sampleresults.
> >     
> >     Bugzilla Id: 64196
> > ---
> >  .../jmeter/testelement/AbstractScopedTestElement.java | 19
> > ++++++++++++++++++-
> >  xdocs/changes.xml                                     |  1 +
> >  2 files changed, 19 insertions(+), 1 deletion(-)

Comment 15 Wicky Tam 2020-03-15 03:40:42 UTC

Thanks for your support!

ok, I'll submit 2 bugs.!

Comment 16 The ASF infrastructure team 2022-09-24 20:38:19 UTC

This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/5252