Bug 63015 - Abnormal NoHttpResponseException when running request through proxy HTTP(S) Test Script Recorder after a first failing request
Summary: Abnormal NoHttpResponseException when running request through proxy HTTP(S) T...
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: HTTP (show other bugs)
Version: 4.0
Hardware: All All
: P2 normal (vote)
Target Milestone: JMETER_5.1
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2018-12-18 09:29 UTC by Philippe Mouawad
Modified: 2018-12-18 15:55 UTC (History)
1 user (show)



Attachments
Reproducer (12.75 KB, application/xml)
2018-12-18 09:31 UTC, Philippe Mouawad
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Mouawad 2018-12-18 09:29:43 UTC

    
Comment 1 Philippe Mouawad 2018-12-18 09:31:32 UTC
Created attachment 36334 [details]
Reproducer

Reproducer:

- Start proxy
- Run plan:
 1) First request is OK
 2) Second request gives abnormal:

org.apache.http.NoHttpResponseException: jmeter.apache.org:80 failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:286)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:257)
	at org.apache.jmeter.protocol.http.sampler.hc.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:199)
	at org.apache.jmeter.protocol.http.sampler.MeasuringConnectionManager$MeasuredConnection.receiveResponseHeader(MeasuringConnectionManager.java:212)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:684)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:697)
	at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:455)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1189)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1178)
	at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:490)
	at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:416)
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:250)
	at java.lang.Thread.run(Thread.java:748)
Comment 2 Felix Schumacher 2018-12-18 09:49:17 UTC
This seems to be a case where the settings of httpclient4.retrycount and httpclient4.request_sent_retry_enabled bites us. Setting them to

httpclient4.retrycount=1
httpclient4.request_sent_retry_enabled=true

or disabling keepalive on the request samplers will get the requests done correctly.

We could probably argue about setting retrycount to 1 as it was a default before 3.0 (see 59103), but I don't think we should enable retrying non-idempotent requests by default.
Comment 3 Philippe Mouawad 2018-12-18 10:09:26 UTC
I found that if we remove the line:

https://github.com/apache/jmeter/blob/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java#L1180

Then it works.

I had compared a HC4 test case with JMeter and the only difference is this header being added.

I remember a discussion somewhere saying that we should not be sending this header anymore.

Do you think it's a HC4 issue or JMeter issue ?

Thanks
Comment 4 Philippe Mouawad 2018-12-18 10:24:19 UTC
(In reply to Philippe Mouawad from comment #3)
> I found that if we remove the line:
> 
> https://github.com/apache/jmeter/blob/trunk/src/protocol/http/org/apache/
> jmeter/protocol/http/sampler/HTTPHC4Impl.java#L1180
> 
> Then it works.
> 
> I had compared a HC4 test case with JMeter and the only difference is this
> header being added.
> 
> I remember a discussion somewhere saying that we should not be sending this
> header anymore.
> 
> Do you think it's a HC4 issue or JMeter issue ?
> 
> Thanks

Ignore my comment, it's stupid.

There is still a difference. 
Comparing logs with HC4 only where I have added Connection:keep-alive , I notice HC4 discards the connection, not JMeter which explains why it works with HC4.
But now I have to find why it is discarded and if it's regular.
Comment 5 Philippe Mouawad 2018-12-18 13:13:36 UTC
I reproduced issue with HC4:

    
    @Test
    public void bugWithPostAndProxy() throws Exception {
        PoolingHttpClientConnectionManager pHCCM = new PoolingHttpClientConnectionManager(2000, TimeUnit.MILLISECONDS);
        pHCCM.setDefaultMaxPerRoute(5);
        pHCCM.setValidateAfterInactivity(1700);
        HttpHost proxy = new HttpHost("localhost", 8888, "http");

        RequestConfig config = RequestConfig.custom().setProxy(proxy).build();

        try (CloseableHttpClient httpclient = HttpClients.custom().
                setConnectionManager(pHCCM).
                setDefaultSocketConfig(SocketConfig.DEFAULT).
                setConnectionTimeToLive(2000, TimeUnit.MILLISECONDS).
                setRetryHandler(new StandardHttpRequestRetryHandler(0, false)).
                setConnectionReuseStrategy(DefaultClientConnectionReuseStrategy.INSTANCE).
                build()) {
            HttpHost target = new HttpHost("jmeter.apache.org");
            HttpPost request = new HttpPost("/404.html");
            request.addHeader("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
            request.addHeader("Connection", "keep-alive");
            request.setConfig(config);
            HttpContext httpContext = new BasicHttpContext();
            httpContext.setAttribute(HttpClientContext.USER_TOKEN, "Thread Group1-1");
            try (CloseableHttpResponse response = httpclient.execute(target, request, httpContext)) {
                logger.info("Headers:{}", response.getEntity().toString());
                EntityUtils.consumeQuietly(response.getEntity());
                logger.info("Headers:{}", response.getAllHeaders());
            }
            request = new HttpPost("/404.html");
            request.addHeader("Connection", "keep-alive");
            try (CloseableHttpResponse response = httpclient.execute(target, request, httpContext)) {
                EntityUtils.consumeQuietly(response.getEntity());
                logger.info("Headers:{}", response.getEntity().toString());
                logger.info("Headers:{}", response.getAllHeaders());
            }
        } 
    }
Comment 6 Philippe Mouawad 2018-12-18 15:23:38 UTC
So issue is on JMeter side as per Oleg help.

To run a real proxy using Docker and squid:
https://github.com/apache/httpcomponents-client/blob/master/httpclient5-testing/docker/BUILDING.txt


git clone httpclient repository
then go to httpcomponents-client/httpclient5-testing/docker

---
cd apache-httpd
sudo docker build -t httpclient-tests-httpd .
sudo docker run --name my-httpclient-tests-httpd -p 0.0.0.0:8080:8080 -p 0.0.0.0:8443:8443 -d httpclient-tests-httpd:latest
---

= Squid 3.3 container

Remark: omit sudo command if executing as root
---
cd squid
sudo docker build -t httpclient-tests-squid .
sudo docker run --name my-httpclient-tests-squid --link my-httpclient-tests-httpd:test-httpd -p 0.0.0.0:8888:8888 -p 0.0.0.0:8889:8889 -d httpclient-tests-squid:latest
---



Indeed it works.

I think the fix is probably to disable KeepAlive when we sample the created request from Proxy.

Thoughts ?
Comment 7 Philippe Mouawad 2018-12-18 15:55:15 UTC
Author: pmouawad
Date: Tue Dec 18 15:54:07 2018
New Revision: 1849188

URL: http://svn.apache.org/viewvc?rev=1849188&view=rev
Log:
Bug 63015 - Abnormal NoHttpResponseException when running request through proxy HTTP(S) Test Script Recorder after a first failing request

Contributed by Ubik Load Pack support
Bugzilla Id: 63015

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/proxy/Proxy.java
    jmeter/trunk/xdocs/changes.xml
Comment 8 Philippe Mouawad 2018-12-18 15:55:43 UTC
Hello,
I have committed my proposal, please test and revert if you have a better idea.

Regards