Bug 52492 - requests with gzip+chunked encoded body return 400 in some cases
Summary: requests with gzip+chunked encoded body return 400 in some cases
Status: RESOLVED DUPLICATE of bug 46146
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_deflate (show other bugs)
Version: 2.2.21
Hardware: PC Linux
: P2 normal with 3 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-01-20 22:54 UTC by Luke Meyer
Modified: 2014-01-28 16:28 UTC (History)
1 user (show)



Attachments
chunk-zip-test.pl script and chunk-test-file.txt for exercising the server; 400.trace shows results; other logs etc. (71.90 KB, application/octet-stream)
2012-01-20 22:54 UTC, Luke Meyer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Meyer 2012-01-20 22:54:21 UTC
Created attachment 28182 [details]
chunk-zip-test.pl script and chunk-test-file.txt for exercising the server; 400.trace shows results; other logs etc.

When a client submits a request body with gzip compression and chunked transfer-encoding, and mod_deflate is directed to unzip input, if the last chunk is "too small" (8 bytes or smaller, as far as I can tell) the server will reject the request with a 400 error. I've observed this when it wasn't the last chunk that's small, but that doesn't seem as repeatable.

Attached are the instruments to reliably produce this problem. It's a little tricky to get a client that will chunk exactly as desired, so I created one manually in Perl - open to the possibility that my script is wrong, but Wireshark seems to understand the requests made, and httpd too depending on the conditions, so I think it's right.

Contents:
chunk-zip-test.pl - Perl test script; requires IO::Socket::INET and Compress::Zlib
chunk-test-file.txt - test content to send (portion of system dictionary)
httpd.conf.snippet - conf to activate mod_deflate (assuming it's loaded/compiled in)
400.trace - example packet trace demonstrating the problem
httpd-V.txt and system - info about my setup
logs.snippet - debug output from bad request
*_log - full logs for context

If httpd is running at localhost:80 with the mod_deflate snippet in place, testing should be very simple:
$ ./chunk-zip-test.pl -size=40
The script defaults gzip and send the file to localhost:80; specifying 40-byte chunks results in the packet trace above with a 400 response.

This particular bug might seem an unlikely use case, but in testing with this script I've noticed a number of possible problem areas that I'll reserve for other bugs, probably related to the central pattern of mod_deflate handling chunked requests. Since this one is relatively simple, it may be the best one to look at first.

I've only tested on Linux (CentOS 5) but have observed the problem with both httpd 2.2.21 and 2.3.16 as well as earlier versions. Let me know if there's anything I can do to assist with this bug.
Comment 1 Lars Tuschinski 2013-01-28 22:10:01 UTC
We are actually seeing this issue without having chunked transfer-encoding. There is just a TCP-packet of 8 bytes size that is not seen by mod_deflate. Those 8 bytes contain the crc and original (unzipped) lenght.

I had a look at the area where I believe the problem shows up. But couldn't find a solution how to read the missing 8 bytes from the wire (as suggested in the FIXME-comment on line 944 in mod_deflate.c of httpd 2.2.23.

But if I can be of any assistance, please let me know.
Comment 2 Christian Nols 2014-01-28 16:28:48 UTC

*** This bug has been marked as a duplicate of bug 46146 ***