Bug 65034 - BinaryTCPClientImpl doesnt skip EOL byte check even when it is set to skip
Summary: BinaryTCPClientImpl doesnt skip EOL byte check even when it is set to skip
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.4
Hardware: All All
: P2 normal (vote)
Target Milestone: JMETER 5.4.1
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2020-12-28 10:10 UTC by hks
Modified: 2021-02-05 06:34 UTC (History)
1 user (show)



Attachments
1a-Sampler (117.20 KB, image/jpeg)
2020-12-28 17:16 UTC, hks
Details
1b-Sampler (135.42 KB, image/jpeg)
2020-12-28 17:17 UTC, hks
Details
1a-Sampler-Response (128.49 KB, image/jpeg)
2020-12-28 17:17 UTC, hks
Details
1b-Sampler-Response (134.87 KB, image/jpeg)
2020-12-28 17:18 UTC, hks
Details
1b-sampler- JMeter2.5.1 (151.78 KB, image/jpeg)
2020-12-29 04:44 UTC, hks
Details
1b-Sampler- Response- JMeter2.5.1 (183.72 KB, image/jpeg)
2020-12-29 04:45 UTC, hks
Details
Jmeter.log file from Jmeter 2.5.1 (6.07 KB, text/plain)
2020-12-29 09:18 UTC, hks
Details
Results-Tab Jmeter 5.4 1b Sampler (163.11 KB, image/jpeg)
2020-12-29 09:53 UTC, hks
Details
Jmeter log 5.4 (6.16 KB, text/plain)
2020-12-29 09:53 UTC, hks
Details
Swallow SocketTimeoutException when no EOM Byte is set (1.87 KB, patch)
2020-12-29 09:56 UTC, Felix Schumacher
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description hks 2020-12-28 10:10:00 UTC
We have TCP sampler requests generated for one of our application in Unisys. the requests sends few bits of data and receive few bits of data (application sends back different response for same request). We had no issue while runnung the same on JMeter 2.5.1 but recently we moved all our scripts to JMeter 5.4 and all our TCP requests are failing. the cause to this what we identified after a thorough testing was that the EOL byte field which needs to be set if we know the response else we can leave it blank as "BinaryTCPClientImpl" class will take 1000 as default which sets the flag to not to check EOL but in reality this is not working. We tried our samples from JMeter initial version where EOL was introduced. From there till 5.4 we saw the same issue. 
Below is the snippet from the src for "BinaryTCPClientImpl". The issue what we are suspecting is the last "IF" statement which is not working as expected.
We also verified running JMeter 5.4 and wireshark on our samplers to make sure if we are sending and receiving data and it looks that JMeter is receiving the data but fails the script in the final stage during EOL check. Please let us know if there is a workaround or anything we are missing out from our observation.

   /**
     * Reads data until the defined EOM byte is reached.
     * If there is no EOM byte defined, then reads until
     * the end of the stream is reached.
     * Response data is converted to hex-encoded binary
     * @return hex-encoded binary string
     * @throws ReadException when reading fails
     */
    @Override
    public String read(InputStream is, SampleResult sampleResult) throws ReadException {
        ByteArrayOutputStream w = new ByteArrayOutputStream();
        try {
            byte[] buffer = new byte[4096];
            int x = 0;
            boolean first = true;
            while ((x = is.read(buffer)) > -1) {
                if (first) {
                    sampleResult.latencyEnd();
                    first = false;
                }
                w.write(buffer, 0, x);
                if (useEolByte && (buffer[x - 1] == eolByte)) {
                    break;
                }
            }
Comment 1 Felix Schumacher 2020-12-28 12:42:57 UTC
Can you provide a bit more information about the test plan, you used or the content of the packets?

Do you set a EOM byte?

Has the last packet (that JMeter processes) such a EOM byte at the boundary of the buffers used? (Maybe you can share the network captures?)

What OS are you using? What JVM?
Comment 2 hks 2020-12-28 17:10:24 UTC
(In reply to Felix Schumacher from comment #1)
> Can you provide a bit more information about the test plan, you used or the
> content of the packets?
          --> We have bunch of TCP Samplers which sends packets and gets a response for each. We do not check or care what data it responds with. We have couple of TCP Samplers at the beginning which performs sign on.

(Have attached the screen shots too in this. 
"1a Sampler" --> always returns "3e" which is 62 in decimal hance we have used/set EOM Byte field.
"1b Sampler" --> the response data is different always, hence we are not setting the EOM Byte field. You can see that we get a response from the application (view results tree screen shot) and JMeter has captured it but due to the EOM check it fails.)

  
> 
> Do you set a EOM byte?  
          --> NO for the request what we send (1b Sampler as attached in screen shot)
> 
> Has the last packet (that JMeter processes) such a EOM byte at the boundary
> of the buffers used? (Maybe you can share the network captures?) 
           --> we do not set the EOM byte for "1b sampler" as the response is different every time. according to the document it says that if we have different response then we can leave EOM byte field empty so that it will be set to "1000" as default which sets the flag as false for EOM byte check. But we dont see that happening in the "IF" condition which was shared earlier. I will be sharing the screen shots of the samplers but will not be able to provide you anything which you can use it to run and check as it is a proprietary application of Unisys.

 
> 
> What OS are you using? What JVM?  
        --> OS doesnt matter as we tried it on Win 10 and WIN2012R2. Its the same result on both.
openjdk 14.0.2 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.2+12)
OpenJDK Client VM AdoptOpenJDK (build 14.0.2+12, mixed mode, sharing)
Also have tried on Oracle JAVA 8.
Comment 3 hks 2020-12-28 17:16:54 UTC
Created attachment 37655 [details]
1a-Sampler

1a sampler
Comment 4 hks 2020-12-28 17:17:27 UTC
Created attachment 37656 [details]
1b-Sampler

1b-Sampler
Comment 5 hks 2020-12-28 17:17:49 UTC
Created attachment 37657 [details]
1a-Sampler-Response

1a-Sampler-Response
Comment 6 hks 2020-12-28 17:18:11 UTC
Created attachment 37658 [details]
1b-Sampler-Response

1b-Sampler-Response
Comment 7 hks 2020-12-29 04:43:45 UTC
More info on this: We have been using JMeter 2.5.1 all these years and the samplers work perfectly fine (Attached screen shots from that). The same when we run on the newer level ex 5.4.1 then it fails. 

Appreciate your help on this.
 
Regards,
HKS
Comment 8 hks 2020-12-29 04:44:27 UTC
Created attachment 37660 [details]
1b-sampler- JMeter2.5.1

1b-sampler- JMeter2.5.1
Comment 9 hks 2020-12-29 04:45:03 UTC
Created attachment 37661 [details]
1b-Sampler- Response- JMeter2.5.1

1b-Sampler- Response in JMeter2.5.1
Comment 10 Felix Schumacher 2020-12-29 09:04:55 UTC
Can you have a look into your log file jmeter.log? I suspect, that you will find warnings about "Read error: " in there (running JMeter 2.5.1)

The main difference between the older versions and the newer ones, are that the errors are note silently swallowed (or converted to warnings in the log files).

@All: I think, we could probably re-create the old behaviour, when no EOM-Byte is set.
Comment 11 hks 2020-12-29 09:18:14 UTC
(In reply to Felix Schumacher from comment #10)
> Can you have a look into your log file jmeter.log? I suspect, that you will
> find warnings about "Read error: " in there (running JMeter 2.5.1)
> 
> The main difference between the older versions and the newer ones, are that
> the errors are note silently swallowed (or converted to warnings in the log
> files).
> 
> @All: I think, we could probably re-create the old behaviour, when no
> EOM-Byte is set.

Hi Felix Schumacher,

I went through jmeter.log file but didnt find any sort of warning messages nor in info messages for "Read error". I have attached the log file here.
Comment 12 hks 2020-12-29 09:18:59 UTC
Created attachment 37662 [details]
Jmeter.log file from Jmeter 2.5.1

Jmeter.log file from JMeter 2.5.1
Comment 13 Felix Schumacher 2020-12-29 09:26:24 UTC
And could you please add at the jmeter.log file in the newer version, too?

What is the message in the sampler result tab of 1b-Sampler in the newer versions?
Comment 14 Felix Schumacher 2020-12-29 09:43:17 UTC
In my tests, I used netcat to simulate a server:

  date | netcat -l 127.0.0.1 1234

and used a TCP Sampler to read from that 'server'.

In my first try, the test didn't end, as I specified no timeout for the read operation. When I did, I got the following exception in the log file (nightly build):

2020-12-29 10:36:40,937 ERROR o.a.j.p.t.s.TCPSampler: 
org.apache.jmeter.protocol.tcp.sampler.ReadException: 
	at org.apache.jmeter.protocol.tcp.sampler.BinaryTCPClientImpl.read(BinaryTCPClientImpl.java:143) ~[ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
	at org.apache.jmeter.protocol.tcp.sampler.TCPSampler.sample(TCPSampler.java:398) [ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
	at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638) [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
	at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
	at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_201]
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_201]
	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_201]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_201]
	at java.net.SocketInputStream.read(SocketInputStream.java:127) ~[?:1.8.0_201]
	at org.apache.jmeter.protocol.tcp.sampler.BinaryTCPClientImpl.read(BinaryTCPClientImpl.java:126) ~[ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
	... 6 more

The end of stream is signalled with a SocketTimeoutException (which is an IOException). In earlier versions, we swallowed the SocketTimeoutException silently. In the newer version, we convert them to a ReadException.

This is, what is hitting you (I suspect). Now, the question remains, what to do here.

I think, we can swallow the SocketTimeoutException in the case, when no EOM-Byte is set. That way, your issue should be gone and those, who can use an EOM-Byte will get notified, when the stream ended early.
Comment 15 hks 2020-12-29 09:52:42 UTC
(In reply to Felix Schumacher from comment #14)
> In my tests, I used netcat to simulate a server:
> 
>   date | netcat -l 127.0.0.1 1234
> 
> and used a TCP Sampler to read from that 'server'.
> 
> In my first try, the test didn't end, as I specified no timeout for the read
> operation. When I did, I got the following exception in the log file
> (nightly build):
> 
> 2020-12-29 10:36:40,937 ERROR o.a.j.p.t.s.TCPSampler: 
> org.apache.jmeter.protocol.tcp.sampler.ReadException: 
> 	at
> org.apache.jmeter.protocol.tcp.sampler.BinaryTCPClientImpl.
> read(BinaryTCPClientImpl.java:143) ~[ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
> 	at
> org.apache.jmeter.protocol.tcp.sampler.TCPSampler.sample(TCPSampler.java:
> 398) [ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
> 	at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638)
> [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
> 	at
> org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.
> java:558) [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
> 	at
> org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489)
> [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
> 	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256)
> [ApacheJMeter_core.jar:5.4.1-SNAPSHOT]
> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
> Caused by: java.net.SocketTimeoutException: Read timed out
> 	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_201]
> 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> ~[?:1.8.0_201]
> 	at java.net.SocketInputStream.read(SocketInputStream.java:171)
> ~[?:1.8.0_201]
> 	at java.net.SocketInputStream.read(SocketInputStream.java:141)
> ~[?:1.8.0_201]
> 	at java.net.SocketInputStream.read(SocketInputStream.java:127)
> ~[?:1.8.0_201]
> 	at
> org.apache.jmeter.protocol.tcp.sampler.BinaryTCPClientImpl.
> read(BinaryTCPClientImpl.java:126) ~[ApacheJMeter_tcp.jar:5.4.1-SNAPSHOT]
> 	... 6 more
> 
> The end of stream is signalled with a SocketTimeoutException (which is an
> IOException). In earlier versions, we swallowed the SocketTimeoutException
> silently. In the newer version, we convert them to a ReadException.
> 
> This is, what is hitting you (I suspect). Now, the question remains, what to
> do here.
> 
> I think, we can swallow the SocketTimeoutException in the case, when no
> EOM-Byte is set. That way, your issue should be gone and those, who can use
> an EOM-Byte will get notified, when the stream ended early.



I think you are spot on with what we are facing presently. We get "Read Exception" for 1b sampler. I have attached the "Results Tab" from Jmeter 5.4 & also the Jmeter.log file from that.
Comment 16 hks 2020-12-29 09:53:14 UTC
Created attachment 37663 [details]
Results-Tab Jmeter 5.4 1b Sampler
Comment 17 hks 2020-12-29 09:53:36 UTC
Created attachment 37664 [details]
Jmeter log 5.4
Comment 18 Felix Schumacher 2020-12-29 09:56:45 UTC
Created attachment 37665 [details]
Swallow SocketTimeoutException when no EOM Byte is set

Earlier versions of JMeter swallowed SocketTimeoutException on the BinaryTCPSamplerImpl. Revert to this behaviour, when no EOM Byte is set, as we have no other way to detect the end of the incoming message.
Comment 19 Felix Schumacher 2020-12-29 10:08:10 UTC
Seems, like the change has been done on purpose with https://bz.apache.org/bugzilla/show_bug.cgi?id=52104
Comment 20 hks 2020-12-29 10:17:27 UTC
(In reply to Felix Schumacher from comment #19)
> Seems, like the change has been done on purpose with
> https://bz.apache.org/bugzilla/show_bug.cgi?id=52104

Yeah but doing so, the scenario which we are encountering will never work on latest versions of JMeter.
Comment 21 hks 2020-12-30 07:13:02 UTC
(In reply to Felix Schumacher from comment #18)
> Created attachment 37665 [details]
> Swallow SocketTimeoutException when no EOM Byte is set
> 
> Earlier versions of JMeter swallowed SocketTimeoutException on the
> BinaryTCPSamplerImpl. Revert to this behaviour, when no EOM Byte is set, as
> we have no other way to detect the end of the incoming message.


So how do we have to take it ahead now?
Comment 22 Felix Schumacher 2021-01-09 17:46:27 UTC
Have committed the fix to trunk. We now ignore SocketTimeoutException, when no EOM is set.

Can you try next nightly or trunk build and report back, whether it fixes your issue?

commit 0ee099997764cab6cc603d26c5ec79957aeaeab1
AuthorDate: Wed Dec 30 12:36:51 2020 +0100

    Ignore SocketTimeoutException on BinaryTCPClientImpl, when no EOM Byte is set
    
    Bugzilla Id: 65034
---
 .../protocol/tcp/sampler/BinaryTCPClientImpl.java      | 18 ++++++++++++------
 xdocs/changes.xml                                      |  3 +++
 2 files changed, 15 insertions(+), 6 deletions(-)
Comment 23 hks 2021-01-13 05:02:47 UTC
(In reply to Felix Schumacher from comment #22)
> Have committed the fix to trunk. We now ignore SocketTimeoutException, when
> no EOM is set.
> 
> Can you try next nightly or trunk build and report back, whether it fixes
> your issue?
> 
> commit 0ee099997764cab6cc603d26c5ec79957aeaeab1
> AuthorDate: Wed Dec 30 12:36:51 2020 +0100
> 
>     Ignore SocketTimeoutException on BinaryTCPClientImpl, when no EOM Byte
> is set
>     
>     Bugzilla Id: 65034
> ---
>  .../protocol/tcp/sampler/BinaryTCPClientImpl.java      | 18
> ++++++++++++------
>  xdocs/changes.xml                                      |  3 +++
>  2 files changed, 15 insertions(+), 6 deletions(-)

Sure i will try this and let you know. Sorry for the delay, i am traveling. 18th Jan Monday will be back. Will do it the same day. Thanks for your support.
Comment 24 hks 2021-01-19 09:37:47 UTC
(In reply to Felix Schumacher from comment #22)
> Have committed the fix to trunk. We now ignore SocketTimeoutException, when
> no EOM is set.
> 
> Can you try next nightly or trunk build and report back, whether it fixes
> your issue?
> 
> commit 0ee099997764cab6cc603d26c5ec79957aeaeab1
> AuthorDate: Wed Dec 30 12:36:51 2020 +0100
> 
>     Ignore SocketTimeoutException on BinaryTCPClientImpl, when no EOM Byte
> is set
>     
>     Bugzilla Id: 65034
> ---
>  .../protocol/tcp/sampler/BinaryTCPClientImpl.java      | 18
> ++++++++++++------
>  xdocs/changes.xml                                      |  3 +++
>  2 files changed, 15 insertions(+), 6 deletions(-)

I took the nightly build or trunk build - "5.4.1-SNAPSHOT 7acd031" and tested our script/testplan and it worked very well, its fixed now :) . Thank you for the support and providing a solution.
Comment 25 hks 2021-01-19 09:39:40 UTC
Will this fix go into the main package?
Comment 26 Philippe Mouawad 2021-01-19 09:41:39 UTC
Hello,
Yes JMeter 5.4.1 is in rc2 currently and should be released this week.

Regards
Comment 27 hks 2021-02-05 06:26:36 UTC
(In reply to Philippe Mouawad from comment #26)
> Hello,
> Yes JMeter 5.4.1 is in rc2 currently and should be released this week.
> 
> Regards

Hi,
Appreciate & Thanks a lot for all this, we were happy to get the resolution. Sure we will take the rc2 build.

Regards,
HKS
Comment 28 Philippe Mouawad 2021-02-05 06:34:37 UTC
Hello,
JMeter 5.4.1 has been released with this fix

Regards