Bug 66282 - mod_proxy_https adds "Content-Encoding: chunked" to requests that have no body
Summary: mod_proxy_https adds "Content-Encoding: chunked" to requests that have no body
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy_http2 (show other bugs)
Version: 2.4.53
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-28 13:51 UTC by Rainer Jung
Modified: 2022-09-29 12:47 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rainer Jung 2022-09-28 13:51:34 UTC
When I send a normal GET request, without body, no Transfer-Encoding and no Content-Length, to httpd and proxy it via mod_proxy_http2 to the same server, the proxied request gets "Transfer-Encoding: chunked" added by mod_proxy_http2. 

I think this is problematic, because

- it seems to me, that "Transfer-Encoding: chunked" is not allowed for http/2 (due to its always streaming behavior), and at least it is unexpected for a GET or HEAD request

- the Core Rule Set 3 of mod_security contains a rule that forbids requests without body but with Transfer-Encoding set to chunked

The header is not added when using mod_proxy_http.

Any chance we can get rid of it when proxying a request, that has no body and doesn't bring the header by its own?

Thanks and regards,

Rainer
Comment 1 Stefan Eissing 2022-09-29 12:47:01 UTC
I made some improvements in release v2.0.8 on github of mod_http2. It might put salve on the issue, but it will not fully resolve it.

To elaborate:

1. The mod_security assumption that requests without body MUST NOT have "Transfer-Encoding: chunked" is wrong. In HTTP/1.1 any request might have trailers and the only way to transport trailers is chunked encoding. 
 
 Now, this may not be used by clients we know. However I recently learned that gRPC uses trailers in responses, even when there is no body. 

2. The "Transfer-Encoding: chunked" that mod_security "sees" is not sent by the client via HTTP/2. mod_proxy_http2 also never sends it. Instead, there are situations where mod_http2 needs to *simulate* chunked encoding for Apache httpd internal processing (in 2.4.x).

HTTP/2 request frames for a stream follow the patterns:

i. HEADER* HEADER(EOH,EOS)
ii. HEADER* HEADER(EOH) DATA* DATA(EOS)
iii. HEADER* HEADER(EOH) DATA* HEADER* HEADER(EOS)

(the "EOS" is a flag at a frame indicating this is the last one for the stream. The "EOH" bit is set at the end of request headers).

Until the EOS is seen by the server, it does not know how many DATA and HEADER frames (for trailers) may follow. Even if there is a "Content-Length: 0", there might be trailer frames coming.

In Apache httpd trunk we solved this problem with the new meta buckets, moving the idea that "everything follows HTTP/1.1 rules" to the transcoding layer, away from our request processing.

But 2.4.x still lives in the HTTP/1.1 world and we need to make the HTTP/2 streams somehow digestible for it.

I hope this illustrates the problem. And maybe mod_security may wish to comment on this.

If you discover cases where we could avoid chunked encoding in 2.4.x, I'd be happy to look at them and see how we can improve.