Bug 62852 - HTTP Request Header missing information when using a proxy
Summary: HTTP Request Header missing information when using a proxy
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: HTTP (show other bugs)
Version: 5.0
Hardware: All All
: P1 regression (vote)
Target Milestone: JMETER_5.1
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2018-10-24 08:49 UTC by arnold.strasser
Modified: 2019-01-25 17:40 UTC (History)
2 users (show)



Attachments
Requests without and with proxy (351.47 KB, image/png)
2018-10-24 08:49 UTC, arnold.strasser
Details
Request Defaults Advanced (24.88 KB, image/png)
2018-11-02 09:07 UTC, arnold.strasser
Details
Request Defaults Basic (22.74 KB, image/png)
2018-11-02 09:07 UTC, arnold.strasser
Details
Request Body (18.31 KB, image/png)
2018-11-02 09:08 UTC, arnold.strasser
Details
Request Header (12.16 KB, image/png)
2018-11-02 09:09 UTC, arnold.strasser
Details
Response Body (63.60 KB, image/png)
2018-11-02 09:09 UTC, arnold.strasser
Details
Response Header (26.28 KB, image/png)
2018-11-02 09:10 UTC, arnold.strasser
Details
JMeter Log (20.60 KB, text/plain)
2018-11-02 09:10 UTC, arnold.strasser
Details
Examples (55.42 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2018-11-05 06:35 UTC, arnold.strasser
Details
Log with http-debug level enabled (214.12 KB, text/plain)
2018-11-05 06:36 UTC, arnold.strasser
Details
Simple test plan to reproduce bug (19.52 KB, application/xml)
2018-12-12 17:04 UTC, Felix Schumacher
Details
Revert the replacement of requests by httpclient when CONNECT is used (1.62 KB, patch)
2018-12-14 13:03 UTC, Felix Schumacher
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description arnold.strasser 2018-10-24 08:49:27 UTC
Created attachment 36212 [details]
Requests without and with proxy

As stated above, when we use a proxy to submit our requests via JMeter, the tab "Request Headers" in the "View Results Tree" is missing information. As soon as we turn off the proxy, the header fields are shown again as you can see in the attachment.
Ironically, this doesnt happen to all requests, about 50% still show all the information despite using a proxy.

Regards, 
Arnold
Comment 1 Philippe Mouawad 2018-10-27 13:10:11 UTC
Hello,
By proxy I guess you mean enterprise proxy ?
1/ Can you show how you  configured it ?
2/ Can you show the HTTP request configuration (Basic and Advanced tabs) and show the full content of Request tabs and Response Tabs , please obfuscate the minimum (header value , not key).
3/ Can you attach jmeter.log ?


Thanks
Comment 2 arnold.strasser 2018-11-02 09:07:02 UTC
Created attachment 36232 [details]
Request Defaults Advanced
Comment 3 arnold.strasser 2018-11-02 09:07:27 UTC
Created attachment 36233 [details]
Request Defaults Basic
Comment 4 arnold.strasser 2018-11-02 09:08:38 UTC
Created attachment 36234 [details]
Request Body
Comment 5 arnold.strasser 2018-11-02 09:09:05 UTC
Created attachment 36235 [details]
Request Header
Comment 6 arnold.strasser 2018-11-02 09:09:43 UTC
Created attachment 36236 [details]
Response Body
Comment 7 arnold.strasser 2018-11-02 09:10:03 UTC
Created attachment 36237 [details]
Response Header
Comment 8 arnold.strasser 2018-11-02 09:10:29 UTC
Created attachment 36238 [details]
JMeter Log
Comment 9 arnold.strasser 2018-11-02 09:14:17 UTC
Hi Philipp! 

Yes, I am talking about an enterprise proxy. 

1) We configured it via an "HTTP Request Defaults" element as shown in the screenshots. The IP of the proxy is read from JMeter properties due to the fact it may differ from test to test. This is also true for other request defaults like server and protocol.

2) & 3) Hope thats everything you need to know is in the screenshots/logfile. If not, please feel free to ask! 

Best regards
Arnold
Comment 10 Philippe Mouawad 2018-11-03 22:39:44 UTC
Hello,
Thanks for further information provided BUT:

- Can you please highlight clearly the names of the missing Headers and show for which requests they are missing and for which the are present ?

I guess a Word file might be better than just screenshots.

Although I understand you may not be able to disclose information, I am sure you can at least just change the values (not keys). 

Also please uncomment in log4j2.xml:
<Logger name="org.apache.http" level="debug" />
<Logger name="org.apache.http.wire" level="debug" />


Then run and attach the jmeter.log file.
Comment 11 arnold.strasser 2018-11-05 06:35:33 UTC
Created attachment 36242 [details]
Examples
Comment 12 arnold.strasser 2018-11-05 06:36:20 UTC
Created attachment 36243 [details]
Log with http-debug level enabled
Comment 13 arnold.strasser 2018-11-05 06:37:48 UTC
Hi Philippe, 

i've tried to put as many info as possible into the word document. Please feel free to ask if you need any additional information. 

Regards,
Arnold
Comment 14 Michael Osipov 2018-12-04 16:18:02 UTC
I can confirm this. As soon as a proxy is involved the headers are omitted from the tab view. As soon as I disable the proxy and go to a local server it is all there. Funny thing is that if I enable debug logging HttpClient happily sends all the headers, no matter what.

For now, I will revert back to 4.0. I have raised priority because this issue is severe.

Philippe, how can I assist? I want to sort this out.
Comment 15 Philippe Mouawad 2018-12-04 17:19:21 UTC
Can you provide http wire captures with and without proxy ?

Thanks
Comment 16 Felix Schumacher 2018-12-12 17:04:15 UTC
Created attachment 36323 [details]
Simple test plan to reproduce bug

I think the attached test plan shows the described problem. You have to start the https script recorder -- our proxy -- in order to see the described problem. All headers from the http header manager won't get included in the response sampler.
Comment 17 Philippe Mouawad 2018-12-13 22:27:16 UTC
Thanks for reproducer, it's great.
Comment 18 Felix Schumacher 2018-12-14 13:03:26 UTC
Created attachment 36331 [details]
Revert the replacement of requests by httpclient when CONNECT is used

It looks like httpclient sets a CONNECT request in the context, when we use a proxy and replaces the original request (informational only) with that CONNECT request.

This patch is a hack to revert the original request, when a CONNECT request is detected. I have no idea, if it is correct, but it works for the attached test plan.
Comment 20 Philippe Mouawad 2018-12-18 20:19:29 UTC
(In reply to Philippe Mouawad from comment #19)
> Opened bug:
> 
> - https://issues.apache.org/jira/browse/HTTPCLIENT-1956
> 
> AS per mailing list discussion:
> 
> -
> http://httpcomponents.10934.n7.nabble.com/Strange-behaviour-when-using-Proxy-
> td34776.html

The below bug has been fixed and will be in HttpClient 4.5.7:

- https://issues.apache.org/jira/browse/HTTPCLIENT-1956
Comment 21 Philippe Mouawad 2018-12-23 10:19:45 UTC
Date: Sun Dec 23 10:18:40 2018
New Revision: 1849596

URL: http://svn.apache.org/viewvc?rev=1849596&view=rev
Log:
Bug 62852 - HTTP Request Header missing information when using a proxy

Patch by Felix Schumacher
Bugzilla Id: 62852

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
    jmeter/trunk/xdocs/changes.xml
Comment 22 Philippe Mouawad 2018-12-23 11:35:42 UTC
Hello,
Could you test build:
https://builds.apache.org/job/JMeter-trunk/6993/artifact/trunk/dist/apache-jmeter-r1849600.zip

and confirm issue is fixed.

Thank you
Comment 23 Philippe Mouawad 2018-12-26 12:49:22 UTC
Hello,
Any feedback ?
Thanks
Comment 24 arnold.strasser 2019-01-08 05:40:34 UTC
Hi Philippe! 
I can confirm that the error is gone, tested with r1850583. 
Thank you very much!
Comment 25 Philippe Mouawad 2019-01-25 17:40:57 UTC
Author: pmouawad
Date: Fri Jan 25 17:40:24 2019
New Revision: 1852155

URL: http://svn.apache.org/viewvc?rev=1852155&view=rev
Log:
Bug 62852 - HTTP Request Header missing information when using a proxy

Upgrade to httpclient 4.5.7 and revert workaround
Bugzilla Id: 62852

Modified:
    jmeter/trunk/LICENSE
    jmeter/trunk/build.properties
    jmeter/trunk/eclipse.classpath
    jmeter/trunk/lib/   (props changed)
    jmeter/trunk/lib/aareadme.txt
    jmeter/trunk/res/maven/ApacheJMeter_parent.pom
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java
    jmeter/trunk/xdocs/changes.xml