Bug 59101

Summary: Wrong behavior with no keepalive when using JAVA Implementation (HC4 fixed since JMeter 3.0)
Product: JMeter Reporter: Amichai <amichai2>
Component: HTTPAssignee: JMeter issues mailing list <issues>
Status: NEW ---    
Severity: normal CC: amichai2, p.mouawad
Priority: P2    
Version: 2.11   
Target Milestone: ---   
Hardware: All   
OS: All   
Attachments: wireshark capture

Description Amichai 2016-03-02 19:55:19 UTC
On Oracle JDK 1.8.0_72 64-bit, Kubuntu 15.10, JMeter 2.11-3 package from standard ubuntu repository.

Setup: 5 HTTP Request samplers which are configured identically other than the path (i.e. 5 different resources from the same server). "Use KeepAlive" is unchecked on all of them. For this test, the thread group is configured to use 1 thread and 1 loop (i.e. a single execution).

Expected behavior:
RFC 7230 section 6.6 states that
"A client that sends a "close" connection option MUST NOT send further
requests on that connection (after the one containing "close") and
MUST close the connection after reading the final response message
corresponding to this request."


So the expected behavior is that after each request the connection should be closed, and a new one created for the next request.

Important note: the server in this case is not sending a "Connection: close" header in the response - the RFC recommends it do so, but it is not required. I'm guessing this may affect the client behavior.

Actual Behavior:
All implementations correctly send the "Connection: close" header to the server, however what follows depends on the implementation:

- HttpClient 3 implementation works correctly.

- Java implementation sends a request with the close header, reads the response and immediately sends the next request on the same connection. The server ignores the second request and closes the connection (as it must according to the RFC). The client then opens a new connection and retries the second (failed) request from the previous connection, which now succeeds, reads the response and once again sends the request for the next (3rd) resource on the same connection, and so on until the end. So effectively, each connection contains the proper request + response plus an extra failed request which will be retried on a subsequent connection. This certainly affects performance during a load test, and is incorrect and unnecessary.

- HttpClient 4 implementation sends a request with the close header, reads the response and immediately sends the next request on the same connection, just like the Java implementation. The server ignores the second request and closes the connection (as it must according to the RFC). The client marks the second request as a failure, and opens a new connection in which it requests the next (3rd) request, etc. So effectively, every odd-numbered request succeeds, and every even-numbered request fails. This is a catastrophic failure of the client, which fails 50% of all requests.

In summary, 2 out of the 3 implementations don't handle connections properly when "Use KeepAlive" is unchecked, and the 3rd (HttpClient 3) functions correctly, but to my understanding it is deprecated and scheduled to be removed.
Comment 1 Philippe Mouawad 2016-03-02 20:31:18 UTC
Hi,
First 2.11 is an old version now.

Now for your report:
For 3) see Bug 58583, it is fixed in nightly build. Feedback welcome:
https://builds.apache.org/job/JMeter-trunk/lastSuccessfulBuild/artifact/trunk/dist/

For 2), could you try setting http.java.sampler.retries=1 using nightly build

Could you set on your JVM option:
-Djavax.net.debug=all

Run your test with Java Impl and attach the logs ?

Thanks for report and feedback.
Regards
Comment 2 Amichai 2016-03-04 23:27:56 UTC
Thanks for the quick response :-)

I can confirm that the issue exists only when the server does not reply with a 'Connection: close' header.

The discussion in Bug #58583 appears correct (other than the subject - I'm not sure a server is misbehaving if it doesn't do something that the RFC does not require it to do).

Using the nightly build you linked to, the issue is indeed solved for HttpClient 4 implementation. Great!

As for the Java implementation, the issue is still there. Running jmeter.sh with JVM_ARGS="-Djavax.net.debug=all" doesn't seem to add anything to jmeter.log. Is there anything else that needs to be configured? Am I looking in the right log file? Anything in particular to look for? (sorry, I'm new to JMeter)
Comment 3 Philippe Mouawad 2016-03-04 23:32:53 UTC
(In reply to Amichai from comment #2)
> Thanks for the quick response :-)
> 
> I can confirm that the issue exists only when the server does not reply with
> a 'Connection: close' header.
> 
> The discussion in Bug #58583 appears correct (other than the subject - I'm
> not sure a server is misbehaving if it doesn't do something that the RFC
> does not require it to do).
> 
> Using the nightly build you linked to, the issue is indeed solved for
> HttpClient 4 implementation. Great!
> 
Thanks for confirmation

> As for the Java implementation, the issue is still there. Running jmeter.sh
> with JVM_ARGS="-Djavax.net.debug=all" doesn't seem to add anything to
> jmeter.log. Is there anything else that needs to be configured? Am I looking
> in the right log file? Anything in particular to look for? (sorry, I'm new
> to JMeter)

Hi, could you use wireshark and attach the capture ?
Thanks
Comment 4 Amichai 2016-03-06 09:54:13 UTC
Created attachment 33633 [details]
wireshark capture

Sure, here it is.
Comment 5 Philippe Mouawad 2016-06-11 08:52:27 UTC
Doesn't it look as a Java Bug as per:
- http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html

@Team, thoughts on this ?

@Amichai, did you try with last Java 9 ?

Thanks
Comment 6 Philippe Mouawad 2017-01-28 13:26:11 UTC
Hello,
Should we open a Java Bug if we do we need to build a simple Test case or just close it as WONTFIX ?
https://docs.oracle.com/javase/8/docs/technotes/guides/net/http-keepalive.html

Regards
Philippe