Bug 61395

Summary: Large server response truncation can impact recording
Product: JMeter Reporter: Alex Podelko <apodelko>
Component: HTTPAssignee: JMeter issues mailing list <issues>
Severity: major CC: p.mouawad
Priority: P2    
Version: 3.1   
Target Milestone: ---   
Hardware: All   
OS: All   
Attachments: the log with debug output

Description Alex Podelko 2017-08-08 22:39:36 UTC
Created attachment 35208 [details]
the log with debug output

Large server response (ab. 11M) got blocked during recording: while the server gets the request and responds, it never gets back to the browser and isn't get recorded by JMeter. Exactly the same request, but with smaller amount of data gets through and gets recorded without any issue.

The end of the server log: - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/comments?_=1501530950881 HTTP/1.1" 200 206 - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/attachments?_=1501530950882 HTTP/1.1" 200 634 - - [31/Jul/2017:12:56:21 -0700] "GET /dmcs/css/images/qual_xls_48_full.png HTTP/1.1" 200 1614 - - [31/Jul/2017:12:56:28 -0700] "GET /dmcs/rest/v1/views/4c819c71-4cb4-4498-ad2d-7c93a3ec6b7d/viewpoints/4944ce44-a073-4e4c-be17-6c77322d8411/nodes/?q=top%7Crequest%3A%3A8aed2d04-ce06-46b9-a86d-91f67fae2114&_=1501530950883 HTTP/1.1" 200 193260 - - [31/Jul/2017:12:56:30 -0700] "GET /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items?expand=item.validations&_=1501530950880 HTTP/1.1" 200 11771576 - - [31/Jul/2017:12:56:33 -0700] "GET /dmcs/rest/v1/views/4c819c71-4cb4-4498-ad2d-7c93a3ec6b7d/viewpoints/4944ce44-a073-4e4c-be17-6c77322d8411/nodes/85204606-a645-4139-a4e0-2a78d0a0b2e3/relationships/06704e43-725b-4266-baae-4e28726e09c8?q=request%3A%3A8aed2d04-ce06-46b9-a86d-91f67fae2114&expand=requestItem.validations&_=1501530950884 HTTP/1.1" 200 10261 

The  request blocked is /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items?expand=item.validations&_=1501530950880

See the log with the debug output attached.
Comment 1 Alex Podelko 2017-08-18 00:31:59 UTC

setting in user.properties
httpsampler.max_bytes_to_ store_per_request=20000000

did the trick - the request got recorded and the server response got back to the browser.

Thank you very much!

Comment 2 Philippe Mouawad 2017-08-27 13:57:22 UTC
Hello Alex,
I am not able to reproduce the non creation of HTTP Request even when response is truncated.

Could you test again to ensure that the request with truncated is really not created.

Comment 3 Alex Podelko 2017-09-05 16:14:06 UTC
Yes, it looks like the request got recorded - sorry, didn't recognize it in Jmeter. So the issue is that it block the server response, so next requests/steps can not be recorded. 

    <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="237 /dmcs/rest/v1/requests/8aed2d04-ce06-46b9-a86d-91f67fae2114/items" enabled="true">
              <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" enabled="true">
                <collectionProp name="Arguments.arguments">
                  <elementProp name="expand" elementType="HTTPArgument">
                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
                    <stringProp name="Argument.name">expand</stringProp>
                    <stringProp name="Argument.value">item.validations</stringProp>
                    <stringProp name="Argument.metadata">=</stringProp>
                  <elementProp name="_" elementType="HTTPArgument">
                    <boolProp name="HTTPArgument.always_encode">false</boolProp>
                    <stringProp name="Argument.name">_</stringProp>
                    <stringProp name="Argument.value">1501168109859</stringProp>
                    <stringProp name="Argument.metadata">=</stringProp>
Comment 4 Philippe Mouawad 2017-09-05 20:43:29 UTC
Hello Alex,
Thanks for your feedback.
So I understand the remaining issue is the following:
- When recording we should never truncate response (to avoid browser error when reading truncated response ) that would break recording next steps

Is this ok for you ?

Comment 5 Alex Podelko 2017-09-05 22:17:09 UTC
yes, looks like a good description of the issue.
Comment 6 Philippe Mouawad 2017-09-07 21:04:17 UTC
Author: pmouawad
Date: Thu Sep  7 20:57:35 2017
New Revision: 1807649

URL: http://svn.apache.org/viewvc?rev=1807649&view=rev
Bug 61395 - Large server response truncation can impact recording
Bugzilla Id: 61395

Comment 7 Philippe Mouawad 2017-09-07 21:04:56 UTC
Hi Alexander, 
Issue should be fixed in next nightly build.
Can you give it a try and give your feedback.
Comment 8 Alex Podelko 2017-09-08 20:02:35 UTC
Works fine with apache-jmeter-r1807728 in the default config - no issues noticed during recording.

Comment 9 Philippe Mouawad 2017-09-08 20:21:33 UTC
Thanks for feedback.

Comment 10 Philippe Mouawad 2017-09-12 20:27:29 UTC
Author: pmouawad
Date: Tue Sep 12 20:26:38 2017
New Revision: 1808160

URL: http://svn.apache.org/viewvc?rev=1808160&view=rev
Bug 61395 - Large server response truncation can impact recording
Truncating responses over 10MB by default was a bad idea.
It is better to disable it by default and have the option available if needed.
Bugzilla Id: 61395