Created attachment 33256 [details] The JMeter configuration + wireshark capture of network traffic Hi, I have an HTTP server I am testing. The particular test pattern has the following behaviour (where some details are excluded for brevity) client> GET /path/to/file [sent to server.example.org] server> 302 Found [Location: http://other.example.org:port/path/to/file] client> GET /path/to/file [sent to other.example.org] server> 200 OK [with the entity as the response body] That operation is repeated several times (using a Loop Controller), but the Thread Group has a single thread, so the requests are serialised. In addition, I placed a one second delay in the loop (using a Constant Timer), so there is at least a one second delay between successive GET requests to other.example.org. I've configured JMeter NOT to reuse connections (the "Use KeepAlive" checkbox is cleared). This means that the client includes the `Connection: close` header in the requests. Both server.example.org and other.example.org close the connection after sending the response. The other point to note is that the port number of other.example.org is always the same. Using wireshark, I have observed that JMeter never reacts to other.example.org sending the FIN packet: there is a corresponding ACK from the client (generated by the OS), but no FIN. This leaves the TCP connection in the half-closed state: CLOSE_WAIT. This is in contrast with JMeter's behaviour with server.example.org, where it does react to FIN and sends its own FIN, fully closing the TCP connection. I have also observed JMeter attempting to reuse these half-closed connections and send a GET request on a CLOSE_WAIT connection. Since the server has closed both ends of the socket, the server OS replies with a RST. This forcefully closes the connection. JMeter appears to have two modes of operation: one where it does not reuse connections and one where it does. When JMeter is not attempting to reuse connections with other.example.org, fresh connections are made to the server for each request. This is despite the existing connection not being closed on the client. When operating like this, all GET requests to other.example.org succeed. When attempting to reuse connections, the client sends a GET request on an existing connection (which is in CLOSE_WAIT state). The server's OS responds with a RST. This RST results in the next iteration's GET request to other.example.org creating a fresh TCP connection. The subsequent GET request succeeds; however, it includes the "Connection: close" header, with the server closing the connection. In the next iteration, the GET request to other.example.org will then fail (RST). Thus, when attempting to reuse connections, the GET requests to other.example.org will alternate between succeeding and failing. I have observed JMeter when running the Test Plan several times. In some cases JMeter initially does not attempt to reuse connections and, after several iterations, it starts attempting to reuse connections. For some Test Plan runs, JMeter always attempts to reuse connections. However, once JMeter has started attempting to reuse connections, I have not seen it revert to not attempting to reuse connections. I haven't been able to determine what factors control this behaviour. Apart from being undesirable behaviour, this is clearly a violation of RFC 2616: where section 8.1.2 "Once a close has been signaled, the client MUST NOT send any more requests on that connection." I'd be grateful if someone could investigate. I have attached the JMeter configuration and a copy of the network traffic (where both client, server.example.org and other.example.org is 'localhost'). Cheers, Paul.
Hello, Could you test with nightly build and check what's happening now ? See Bug 57921 Thanks
Tested with latest nightly build (r1710693) and the problem persists. From its description, bug 57921 is unrelated: that bug refers to a problem when "Keep Alive" enabled. This bug is about a problem with "Keep Alive" switched off.
Supporting the OPs request: RFC 7230 also says: 6.6. Tear-down ... A client that sends a "close" connection option MUST NOT send further requests on that connection (after the one containing "close") and MUST close the connection after reading the final response message corresponding to this request. In this case here, the the requests to the first and the second server contains the client side "Connection: close", but only the first server responds with "Connection: close". The second does not, but both close the connection. Although it is not nice from the second server to not send "Connection: close", it is allowed (same RFC section: "The server SHOULD send a "close" connection option in its final response on that connection."). Looking at HTTPHC3Impl and HTTPHC4Impl it really seems we only set the "Connection: close" header but don't tell the HTTP clent to actually close the connection. I didn't check for the Java impl. So there really seems to be a bug here.
You're quite correct in noting that the second server failed to send the "Connection: close" header in the response. Independent of this problem, that should be fixed.
Hi, Isn't this issue: https://issues.apache.org/jira/browse/HTTPCLIENT-1655 @Paul Millar, we upgraded today to HC4.5.1. Could you try nightly build tomorrow and give us some feedback ? Thanks
(In reply to Philippe Mouawad from comment #5) > Hi, > Isn't this issue: > https://issues.apache.org/jira/browse/HTTPCLIENT-1655 > > @Paul Millar, we upgraded today to HC4.5.1. > Could you try nightly build tomorrow and give us some feedback ? > Thanks Forget my last comment. I misunderstood the problem. Now let me recap what I understand: The issue is due to server not setting header "Connection: close", and JMeter as a side effect not closing the connection while it should . Is my understanding correct ? Thanks
Yes, that was my impression when I looked at the ticket. I wrote "Looking at HTTPHC3Impl and HTTPHC4Impl it really seems we only set the "Connection: close" header but don't tell the HTTP clent to actually close the connection. I didn't check for the Java impl." If the *server* sends "Connection: close", then the client will automatically close the ocnnection, but only sending a "Connection: close" from the client side does not lead to automaticaly closing it in the client. IMHO we have to tell the client not only to send "Connection: close" but also to close the connection. Most servers do send "Connection: close" so often the problem doesn't occur.
Hi, It appears it was an issue in HTTPCORE: https://issues.apache.org/jira/browse/HTTPCORE-397 Could you please test nightly build and report wether issue is fixed for you ? Thanks
Hi, New update , fix is not in HTTPcore-4.4.4 so no need to test. Regards Philippe
Fix should be available in HTTPClient 4.5.2 as per commit: - https://github.com/ok2c/httpclient/commit/8f38a042494482e18f0129baf4f5118067acedde To have it in place, we will need to update HTTPHC4Impl to use the DefaultClientConnectionReuseStrategy.INSTANCE: ((AbstractHttpClient) httpClient).setReuseStrategy(DefaultClientConnectionReuseStrategy.INSTANCE);
Author: pmouawad Date: Fri Feb 26 22:35:18 2016 New Revision: 1732569 URL: http://svn.apache.org/viewvc?rev=1732569&view=rev Log: Upgrade to HttpClient/httpmime 4.5.2 which fixes at the same time: - Bug 58881 - HTTP Request : HTTPHC4Impl shows exception when server uses "deflate" compression - Bug 58583 - HTTP client fails to close connection if server misbehaves by not sending "connection: close", violating HTTP RFC 2616 / RFC 7230 - Bug 56358 - Cookie manager supports cross port cookies and RFC6265 - Bug 57319 - Upgrade to HttpClient 4.5.2 Bugzilla Id: 57319 Modified: jmeter/trunk/build.properties jmeter/trunk/eclipse.classpath jmeter/trunk/lib/ (props changed) jmeter/trunk/lib/aareadme.txt jmeter/trunk/licenses/bin/README.txt jmeter/trunk/res/maven/ApacheJMeter_parent.pom jmeter/trunk/test/src/org/apache/jmeter/protocol/http/control/TestCookieManager.java jmeter/trunk/test/src/org/apache/jmeter/protocol/http/sampler/TestHTTPSamplersAgainstHttpMirrorServer.java jmeter/trunk/xdocs/changes.xml
Author: pmouawad Date: Fri Feb 26 22:50:37 2016 New Revision: 1732574 URL: http://svn.apache.org/viewvc?rev=1732574&view=rev Log: Bug 58950 - NoHttpResponseException when Pause between samplers exceeds keepalive sent by server Bugzilla Id: 58950 Modified: jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java jmeter/trunk/xdocs/changes.xml
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/3688