Bug 50275 - Mod_JK does not set chunk size for response with chunked transfer encoding
Summary: Mod_JK does not set chunk size for response with chunked transfer encoding
Status: RESOLVED INVALID
Alias: None
Product: Tomcat Connectors
Classification: Unclassified
Component: mod_jk (show other bugs)
Version: 1.2.27
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-15 14:12 UTC by Andre Piwoni
Modified: 2010-11-17 14:20 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andre Piwoni 2010-11-15 14:12:23 UTC
Mod_JK does not handle correctly the case when Transfer-Encoding header is set to chunked on HTTP response returned from Tomcat via AJP. Transfer-Encoding is set on HttpServletResponse and no Content-Length header is set. In this scenario  Mod_JK returns response to client with Transfer-Encoding set to chunked, but it fails to set chunk-size before the body. When the same response is returned with no Transfer-Encoding and no Content-Length set, Mod_JK sets Transfer-Encoding to chunked, sets the chunk size and returns response to the client.

Transfer-Encoding is a hop-to-hop header and does not have to be set on the response from Mod_JK to client, but when it is I would expect Mod_JK
set the chunk size.

Server   Apache/2.2.11 (Win32) mod_ssl/2.2.11 OpenSSL/0.9.8i mod_jk/1.2.27
Apache Tomcat 5.5.30 using AJP/1.3.

Also, happens on SunOS and with other versions of Tomcat and with Apache 2.2.14.
Comment 1 Tim Whittington 2010-11-17 03:13:35 UTC
This doesn't sound like a Tomcat bug:
- The AJP connector in Tomcat (on the Java side) does not do chunked encoding - this is done by the web server (in the case of Apache - i.e. not mod_jk) or the ISAPI Redirector (in the case of IIS).
- If you're setting Transfer-Encoding: chunked in a servlet, then everything else in the response chain back to the browser will assume that you're also chunk encoding the response body, otherwise how would Tomcat know whether you'd chunk encoded the body or it had to? 
For comparison the DefaultServlet in Tomcat does HTTP byte range responses (which have similar implications for response encoding and message length detection to chunked encoding) and Tomcat and mod_jk behave well with that.

There's also the question of why you'd want to explicitly set Transfer-Encoding: chunked on a response when every available connector exposing your content to the client supports it already without you having to do anything...

If I'm misunderstanding the situation, please attach a sample servlet and mod_jk DEBUG + HTTP trace that illustrates the problem.
Comment 2 Andre Piwoni 2010-11-17 14:20:21 UTC
Tim,

Few questions come to mind:

(1) If this is not Apache/mod_jk bug, are you saying this is AJP connector bug?
(2) If none of the above, are you saying that setting Transfer-Encoding in servlet is illegal?
(3) If setting Transfer-Encoding is legal in servlet, do you think that servlet should be aware of its container and limitations, if any, of specific connectors like no chunked encoding on AJP? After all, if I set chunked encoding for mod_proxy or without any proxy it is working for HTTP connector.

Please see further comments below:

(In reply to comment #1)
> This doesn't sound like a Tomcat bug:
> - The AJP connector in Tomcat (on the Java side) does not do chunked encoding 
> this is done by the web server (in the case of Apache - i.e. not mod_jk) or the ISAPI Redirector (in the case of IIS).

Tomcat HTTP connector appears to be handling Transfer-Encoding set by servlet to chunked. Again, servlet should not be aware of container specifics (AJP/Mod_proxy or no proxy etc.) other than knowing that client can handle chunked Transfer-Encoding. I strongly disagree that this determination should be made based on connector type. Rather it should be made based on criteria defined in HTTP specs. AJP is passing Transfer-Encoding set to chunked to mod_jk and things break. Perhaps, AJP SHOULD transparently remove this header if it does not handle it or mod_jk could remove it. Please keep in mind Transfer-Encoding is a hop-to-hop header. It does not have to be propagated all the way via proxies to the client.

> - If you're setting Transfer-Encoding: chunked in a servlet, then everything
> else in the response chain back to the browser will assume that you're also
> chunk encoding the response body, otherwise how would Tomcat know whether you'd chunk encoded the body or it had to? 

That is not entirely correct or maybe entirely incorrect. I'm setting Transfer-Encoding to chunked in servlet and don't do actual chunking, yet when I use client to directly access this servlet (no proxy) via HTTP connector the response magically seems to be chunked. Are you saying I should do actual chunking for AJP and no chunking for HTTP connectors? Consistency is my main problem here! Is this the case of other containers when servlet has to do actual chunking? I thought that this was responsibility of the container.
As a side note, any proxy along the response chain can terminate chunked Transfer-Encoding if its client is not able to handle it.

> For comparison the DefaultServlet in Tomcat does HTTP byte range responses
> (which have similar implications for response encoding and message length
> detection to chunked encoding) and Tomcat and mod_jk behave well with that.
> 
> There's also the question of why you'd want to explicitly set
> Transfer-Encoding: chunked on a response when every available connector
> exposing your content to the client supports it already without you having to
> do anything...

I understand that chunked encoding is automatic in many cases, like some compressed files. I also understand that connector or web server in the absence of Content-Length, if client supports chunked Transfer-Encoding, can set it to chunked or may even compute Content-Length for you.
My case is that of third-party software (Mule ESB)that can run stadalone, where HTTP connections can be made directly. In this case, in the absence of Content-Length it may set Transfer-Encoding to chunked and this causes issues because mod_jk/AJP setup does not handle it correctly.

> 
> If I'm misunderstanding the situation, please attach a sample servlet and
> mod_jk DEBUG + HTTP trace that illustrates the problem.

mod_jk debug log shows exactly no difference other than when I set Transfer-Encoding to chunked, the header is returned otherwise there are no differences.
Whether I choose to set Transfer-Encoding or not in the servlet, Apache chooses to set encoding to chunked since there's no Content-Length, but with Transfer-Encoding set there's no chunk size.

Correctly chunked response by Apache has the chunk size set to 119 in this example, and no chunk size when Transfer-Encoding is set in servlet, in which case, client hangs as I would expect. I have tested this in simplified setup so there's nothing special going on here.

"Content-Type: text/xml[\r][\n]"
"Transfer-Encoding: chunked[\r][\n]"
"[\r][\n]"
"1"
"1"
"9"
"[\r]"
"[\n]"
"<soap:Envelope ..."

If I understand you correctly, you are saying that because AJP passes along Transfer-Encoding and does not chunk the actual response,the Apache assumes that response is chunked, including the size, and passes it along to the client.
While this makes sense, Transfer-Encoding is a hop-to-hop header and as such should be removed if not supported by mod_jk/AJP. I don't think this burden should rest on the servlet since (1) it should not be aware of the container it runs in (2) client request indicates HTTP 1.1 is supported so Transfer-Encoding should be fine.

Regards,
Andre Piwoni