Bug 61395

Summary: Large server response truncation can impact recording
Product: JMeter - Now in Github Reporter: Alex Podelko <apodelko>
Component: HTTPAssignee: JMeter issues mailing list <issues>
Status: RESOLVED FIXED    
Severity: major CC: p.mouawad
Priority: P2    
Version: 3.1   
Target Milestone: ---   
Hardware: All   
OS: All   
Attachments: the log with debug output

Description Alex Podelko 2017-08-08 22:39:36 UTC
Created attachment 35208 [details]
the log with debug output

Large server response (ab. 11M) got blocked during recording: while the server gets the request and responds, it never gets back to the browser and isn't get recorded by JMeter. Exactly the same request, but with smaller amount of data gets through and gets recorded without any issue.

The end of the server log:

10.240.164.237 - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/comments?_=1501530950881 HTTP/1.1" 200 206 
10.240.164.237 - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/attachments?_=1501530950882 HTTP/1.1" 200 634 
10.240.164.237 - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/css/images/qual_xls_48_full.png HTTP/1.1" 200 1614 
10.240.164.237 - - [31/Jul/2017:12:56:28 -0700] "GET /dmcs/rest/v1/views/4c819c71-4cb4-4498-ad2d-7c93a3ec6b7d/viewpoints/4944ce44-a073-4e4c-be17-6c77322d8411/nodes/?q=top%7Crequest%3A%3A8aed2d04-ce06-46b9-a86d-91f67fae2114&_=1501530950883 HTTP/1.1" 200 193260 
10.240.164.237 - - [31/Jul/2017:12:56:30 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items?expand=item.validations&_=1501530950880 HTTP/1.1" 200 11771576 
10.240.164.237 - - [31/Jul/2017:12:56:33 -0700] "GET /dmcs/rest/v1/views/4c819c71-4cb4-4498-ad2d-7c93a3ec6b7d/viewpoints/4944ce44-a073-4e4c-be17-6c77322d8411/nodes/85204606-a645-4139-a4e0-2a78d0a0b2e3/relationships/06704e43-725b-4266-baae-4e28726e09c8?q=request%3A%3A8aed2d04-ce06-46b9-a86d-91f67fae2114&expand=requestItem.validations&_=1501530950884 HTTP/1.1" 200 10261 


The  request blocked is /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items?expand=item.validations&_=1501530950880

See the log with the debug output attached.
Comment 1 Alex Podelko 2017-08-18 00:31:59 UTC
Hi, 

setting in user.properties
httpsampler.max_bytes_to_ store_per_request=20000000

did the trick - the request got recorded and the server response got back to the browser.

Thank you very much!

Alex
Comment 2 Philippe Mouawad 2017-08-27 13:57:22 UTC
Hello Alex,
I am not able to reproduce the non creation of HTTP Request even when response is truncated.

Could you test again to ensure that the request with truncated is really not created.

Thanks
Comment 3 Alex Podelko 2017-09-05 16:14:06 UTC
Yes, it looks like the request got recorded - sorry, didn't recognize it in Jmeter. So the issue is that it block the server response, so next requests/steps can not be recorded. 

    <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="237 /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items" enabled="true">
              <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" enabled="true">
                <collectionProp name="Arguments.arguments">
                  <elementProp name="expand" elementType="HTTPArgument">
                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
                    <stringProp name="Argument.name">expand</stringProp>
                    <stringProp name="Argument.value">item.validations</stringProp>
                    <stringProp name="Argument.metadata">=</stringProp>
                  </elementProp>
                  <elementProp name="_" elementType="HTTPArgument">
                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
                    <stringProp name="Argument.name">_</stringProp>
                    <stringProp name="Argument.value">1501168109859</stringProp>
                    <stringProp name="Argument.metadata">=</stringProp>
                  </elementProp>
                </collectionProp>
              </elementProp>
Comment 4 Philippe Mouawad 2017-09-05 20:43:29 UTC
Hello Alex,
Thanks for your feedback.
So I understand the remaining issue is the following:
- When recording we should never truncate response (to avoid browser error when reading truncated response ) that would break recording next steps


Is this ok for you ?

Thanks
Comment 5 Alex Podelko 2017-09-05 22:17:09 UTC
yes, looks like a good description of the issue.
Comment 6 Philippe Mouawad 2017-09-07 21:04:17 UTC
Author: pmouawad
Date: Thu Sep  7 20:57:35 2017
New Revision: 1807649

URL: http://svn.apache.org/viewvc?rev=1807649&view=rev
Log:
Bug 61395 - Large server response truncation can impact recording
Bugzilla Id: 61395

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/threads/JMeterContext.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/proxy/Proxy.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSamplerBase.java
    jmeter/trunk/xdocs/changes.xml
Comment 7 Philippe Mouawad 2017-09-07 21:04:56 UTC
Hi Alexander, 
Issue should be fixed in next nightly build.
Can you give it a try and give your feedback.
Regards
Comment 8 Alex Podelko 2017-09-08 20:02:35 UTC
Works fine with apache-jmeter-r1807728 in the default config - no issues noticed during recording.

Thanks,
Alex
Comment 9 Philippe Mouawad 2017-09-08 20:21:33 UTC
Thanks for feedback.

Regards
Comment 10 Philippe Mouawad 2017-09-12 20:27:29 UTC
Author: pmouawad
Date: Tue Sep 12 20:26:38 2017
New Revision: 1808160

URL: http://svn.apache.org/viewvc?rev=1808160&view=rev
Log:
Bug 61395 - Large server response truncation can impact recording
Truncating responses over 10MB by default was a bad idea.
It is better to disable it by default and have the option available if needed.
Bugzilla Id: 61395

Modified:
    jmeter/trunk/bin/jmeter.properties
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSamplerBase.java
    jmeter/trunk/xdocs/changes.xml
    jmeter/trunk/xdocs/usermanual/properties_reference.xml
Comment 11 The ASF infrastructure team 2022-09-24 20:38:09 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/4463