Bug 23287

Summary: mod_deflate does not properly format decompressed request content
Product: Apache httpd-2 Reporter: John N Armstrong <john>
Component: mod_deflateAssignee: Apache HTTPD Bugs Mailing List <bugs>
Severity: normal CC: john
Priority: P3    
Version: 2.0.47   
Target Milestone: ---   
Hardware: PC   
OS: All   

Description John N Armstrong 2003-09-20 03:36:26 UTC
mod_deflate is configured to decompress an inbound HTTP SOAP request (note: The Apache SOAP TCPTunnel/Monitor was used to capture the flows between the Apache proxy server and WebLogic):

POST /some/url HTTP/1.0
Host: localhost:1080
Content-Type: text/xml; charset=utf-8
Content-Length: 4727
SOAPAction: "http://some.soap.action"
Accept-Encoding: gzip,deflate
Content-Encoding: gzip
<SNIP: Compressed SOAP message/>

The HTTP request body (the SOAP request) is compressed in the gzip format.

mod_deflate does intercept and decompress the content.

Apache is also configured (using mod_proxy) to forward the request on to WebLogic. The request forwarded to WebLogic is as follows:

POST /some/url HTTP/1.1
Host: localhost:7101
Content-Type: text/xml; charset=utf-8
Content-Length: 4727 <== Note!
SOAPAction: "http://some.soap.action"
Accept-Encoding: gzip,deflate
Content-Encoding: gzip <== Note!
<SNIP: Decompressed SOAP message />

mod_deflate should mdoify (or remove) the Content-Encoding header to reflect the fact that the content is NOT compressed, and should re-write the Content-Length header to reflect the actual length of the decompressed content.
Comment 1 André Malo 2003-09-21 19:31:48 UTC
Iit cannot change the content-length header, because the data is streamed. Hmm,
I guess it's not really intended to decompress as a proxy.
Can you try what happens, if you use mod_header's `RequestHeader unset'
directive to remove the headers in question?

I think, mod_proxy should be changed, that if the content-length header is
removed, it switches to chunked encoding.... duh, that is only possible with
HTTP/1.1. Seems there's no simple solution to that problem.

I'm +1 anyway to remove the gzip token from Content-encoding.

Any other ideas?
Comment 2 John N Armstrong 2003-09-22 04:09:39 UTC
Yes, I think that mod_deflate is rarely used for decompression of the REQUEST -- it's almost always used for compression of the RESPONSE. And, the combination of mod_deflate and proxying the request to a back-end server appears to be problematic.

I tried using mod_headers and the RequestHeader unset command to remove the Content-Encoding: gzip and Content-Length: xxxx headers. The problem with this is that it causes mod_deflate to be be nop-op'ed because he doesn't recognize that the incoming request is compressed without the header.

I considered writing a filter to post-process mod_deflate's output and remove the headers, but that won't work either, because the output is being streamed (as you point out), and the headers will already have been written.
Comment 3 Nick Kew 2004-09-05 15:48:35 UTC
Hmmm, ...

Can you not use (the mirror of) the workaround I used to recommend for
mod_proxy_html before updating mod_deflate to support decompression of response
data from a proxy?

Header unset Accept-Encoding

to tell the Client you don't want gzipped data?

I might actually fix this properly, but no promises.
Comment 4 Michael Klepikov 2005-04-19 17:44:29 UTC
A related problem with servlet requests forwarded to Tomcat via mod_jk: request
content gets decompressed, but I get an end of stream at (the original)
Content-Length bytes from stream start, resulting in truncated decompressed
content for the servlet.

I do not insist on removing "gzip" from Content-Encoding or removing
Content-Length. It's completely OK with me to just ignore Content-Length in my
servlet and rely on the servlet request end of stream. Servlets have to be able
to handle chunked requests anyway, so using Content-Length is not good style
regardless of compression. It's also OK with me to ignore "gzip" in
Content-Encoding, as long as I know that Apache handles it for me. But it's not
OK to get truncated content, I obviously do need full decompressed content in
the servlet.
Comment 5 André Malo 2005-04-19 18:04:05 UTC
(In reply to comment #4)
> A related problem with servlet requests forwarded to Tomcat via mod_jk: request

I'm certain, this should be fixed in mod_jk. Could you open a bug report /
feature  request at mod_jk's tracker?
Comment 6 Michael Klepikov 2005-04-19 19:53:19 UTC
Opened http://issues.apache.org/bugzilla/show_bug.cgi?id=34526
Comment 7 Michael Klepikov 2005-08-08 01:46:02 UTC
William Barker from mod_jk marked 34526 "resolved, wontfix" and insists it's a
mod_deflate problem. Would it be possible to have a direct discussion among the
developers responsible for this functionality in mod_deflate and mod_jk? It's
important to get it fixed, wherever the fixes may happen to be...
Comment 8 Nick Kew 2007-07-29 01:46:36 UTC
Fixed in /trunk/ and proposed for backport

This kind of thing needs round tuits.  But it's still been an inexcusably long
time for a real bug.
Comment 9 John N Armstrong 2007-07-30 11:47:23 UTC
Understand. Well, better late than never, eh? I'd long-since moved on to other projects and forgotten 
about this. ;-) Thanks for taking care of it.
Comment 11 Jess Holle 2007-08-07 05:33:11 UTC
Any chance of a backport to Apache 2.2.x?
Comment 12 Ruediger Pluem 2007-08-07 12:30:18 UTC
(In reply to comment #11)
> Any chance of a backport to Apache 2.2.x?

It is already backported to Apache 2.2.x. See Comment 10.