Status: RESOLVED DUPLICATE of bug 65179
Alias: None
Product: Tomcat 9
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 9.0.38
Hardware: PC Linux
: P2 blocker (vote)
Target Milestone: -----
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2021-12-30 13:47 UTC by Arshiya
Modified: 2022-01-04 09:53 UTC (History)
0 users

PCAP (134.19 KB, image/png)
2021-12-30 13:47 UTC, Arshiya
PCAP where requests are not processed with default overhead params (18.07 KB, application/vnd.tcpdump.pcap)
2021-12-31 12:49 UTC, Arshiya
Sample JMX file (117.75 KB, application/xml)
2021-12-31 12:50 UTC, Arshiya
Reproducer (437.17 KB, application/x-zip-compressed)
2021-12-31 12:50 UTC, Arshiya

Note You need to log in before you can comment on or make changes to this bug.
Description Arshiya 2021-12-30 13:47:09 UTC
Created attachment 38150 [details]

Hi All,
We are using embedded tomcat version 9.0.38 in our application to transport http/2 packets between 2 systems (h2c connection). All parameters used are the tomcat defaults.
Tomcat port:1090

Issue :
For few streams we see that :
After processing requests successfully, tomcat does not send WINDOW_UPDATE abruptly for the following HEADERS/DATA, due to which the client sends RST_STREAM, also which tomcat has not acknowledged.

Since the issue has occured in production , we wont be able to test with the latest version on tomcat .

Find attached the PCAP for reference:

Please help resolve this issue.

Thanks in advance !!

Thanks and Regards
Arshiya Shariff
Comment 1 Michael Osipov 2021-12-30 13:53:40 UTC
Upgrade first.
Comment 2 Arshiya 2021-12-30 14:28:10 UTC
The application was working fine with the same version since few months , so we want to find the RCA as to why the issue has occurred . 
Was there any fix around this that went in post 9.0.38 so we can update the same to the customer otherwise it is difficult to push the customer for an upgrade.
Comment 3 EJ Egyed 2021-12-30 14:46:09 UTC
There are several fixes to HTTP/2 window updates between version 9.0.38 and 9.0.56 (current latest). You can review the changes here https://tomcat.apache.org/tomcat-9.0-doc/changelog.html and just do a quick search for HTTP/2.
Comment 4 Christopher Schultz 2021-12-30 17:10:07 UTC
(In reply to Arshiya from comment #2)
> The application was working fine with the same version since few months , so
> we want to find the RCA as to why the issue has occurred . 

It is far easier for you to test this in your environment to see how/if the issue is fixed than it would be for anyone else to test it. If we had tests demonstrating a problem, we would fix the problem so the tests pass.

If you have a test that fails, please attach it to this report. If you do not have a test that fails, it will be impossible for anyone to tell you whether or not it is fixed.
Comment 5 Arshiya 2021-12-31 04:22:32 UTC
As tomcat abruptly stops sending the WINDOW_UPDATE we are unable to reproduce the issue in the test environment and find where the issue is . 
Can we have any scenarios to simulate a case where tomcat suddenly stops sending WINDOW_UPDATE? Please let us know.
Comment 6 Mark Thomas 2021-12-31 10:35:45 UTC
Absent any evidence to the contrary, this issue is assumed to be a duplicate. It should only be re-opened if:
- the issue can be produced with the latest 9.0.x release
- the steps to reproduce the issue can be provided

*** This bug has been marked as a duplicate of bug 65179 ***
Comment 7 Arshiya 2021-12-31 12:49:26 UTC
Created attachment 38151 [details]
PCAP where requests are not processed with default overhead params
Comment 8 Arshiya 2021-12-31 12:50:03 UTC
Created attachment 38152 [details]
Sample JMX file
Comment 9 Arshiya 2021-12-31 12:50:46 UTC
Created attachment 38153 [details]
Comment 10 Arshiya 2021-12-31 12:51:31 UTC
We are trying to reproduce the issue with version 9.0.56 .
Observation :
With window size of 1000 and default values for all other params , we see that few requests are not being processed (PCAP attached) . But on changing overheadContinuationThreshold , overheadCountFactor , overheadDataThreshold , overheadWindowUpdateThreshold to "0" all requests process fine . 

Will changing the 4 overhead params from the default values to zero solve the issue ? Will there be any side effects because of this?

I have attached the JMX , source code used to reproduce the issue where few requests are not processed, though it is not the exact reproducer for the WINDOW_UPDATE issue .
Comment 11 Mark Thomas 2022-01-04 09:53:09 UTC
The original missing WINDOW_UPDATE issuer is a duplicate so I am restoring the resolution of this issue.

That the client is now triggering the overhead protection is not a bug. If you want to explore why this is happening then the users list is the place to do so. If you enable debug logging:

org.apache.coyote.http2.level = FINE

in logging.properties then you'll see the overhead values changing as each frame is received and what is triggering the protection.

*** This bug has been marked as a duplicate of bug 65179 ***