Bug 42608 - Invalid Content-Length error for the binary file size greater than 2.1GB
Invalid Content-Length error for the binary file size greater than 2.1GB
Status: CLOSED FIXED
Product: Tomcat 5
Classification: Unclassified
Component: Connector:AJP
5.5.6
Other Solaris
: P2 normal (vote)
: ---
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2007-06-06 12:23 UTC by Daniel Dang
Modified: 2008-01-01 17:53 UTC (History)
0 users



Attachments
Rough patch for http 2.2.x only (4.27 KB, patch)
2007-06-17 14:35 UTC, Mark Thomas
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Dang 2007-06-06 12:23:46 UTC
Apache-tomcat-5.5.16 version.
Every time when I send 2GB binary data the apache-tomcat-5.5.16 is working fine.
But when I tried to send 3GB binary data following error below: I did traced to
the error code in HandlerRequest.decodeHeaders method called msg.getInt() this
method is a problem because this method is returning an int instead of long. ant
Integer is only handle 2.1GB limit but I am sending more than 3GB. Is there a
way that you can fix this MessageBytes.getLong() instead of getInt()?

I am looking for a solution to send more than 3GB data to apache-tomcat

Many thanks in advance..

Here is an error below.

[TP-Processor3] ERROR common.HandlerRequest  - Error decoding request
java.lang.NumberFormatException
        at org.apache.tomcat.util.buf.Ascii.parseInt(Ascii.java:145)
        at org.apache.tomcat.util.buf.ByteChunk.getInt(ByteChunk.java:521)
        at org.apache.tomcat.util.buf.MessageBytes.getInt(MessageBytes.java:688)
        at org.apache.jk.common.HandlerRequest.decodeHeaders(HandlerRequest.java:57
9)
        at org.apache.jk.common.HandlerRequest.decodeRequest(HandlerRequest.java:39
7)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:260)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:754)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
684)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.
java:876)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool
.java:684)
        at java.lang.Thread.run(Thread.java:619)


thanks

--Daniel
Comment 1 Christopher Schultz 2007-06-11 14:22:25 UTC
Daniel, please post the headers that your web browser is sending to Apache
httpd. Also, please tell us the version of mod_jk (or mod_proxy_ajp) you are using.
Comment 2 Daniel Dang 2007-06-12 07:57:44 UTC
Christ, I don't use web browser to send data to apache httpd. How do I display
the  httpd header? Can you tell me a command how to do that, turn on the debug
or something?

The version of mod_proxy_ajp is Apache/2.2.3
The version of mod_jk is I don't know how to see them?



Comment 3 Christopher Schultz 2007-06-12 08:56:47 UTC
You need to tell us if you are using mod_jk or mod_proxy_ajp, because there is
some question as to which component contains this problem.

Check your Apache httpd configuration for how you connect Apache httpd to
Tomcat. If you are using mod_proxy_ajp, then you will have configuration
directives like "ProxyPass /myApp ajp://localhost:8080/myApp". If you are using
mod_jk, you will have directives such as "JkMount".

If you are using mod_jk, we'll need to know the version number. Sometimes the
name of the module being loaded into Apache httpd has the version number right
in the filename (i.e. mod_jk_1.2.13.so). If not, you can look into the file to
find out what version you have:

$ strings /usr/lib/apache2-extramodules/mod_jk.so | grep "mod_jk/"
[output]
mod_jk/1.2.18

The version I have checked here is 1.2.18. You should upgrade to the latest
version of mod_jk (currently 1.2.23) if possible.
Comment 4 Daniel Dang 2007-06-12 10:16:20 UTC
I believe I am using mod_proxy_ajp connector because this is what I have on my
httpd.conf:
ProxyPass /axis ajp://192.168.11.10:8009/axis
ProxyPassReverse /axis ajp://192.168.11.10:8009/axis

The version of mod_proxy_ajp is Apache/2.2.3
Comment 5 william.barker 2007-06-12 19:02:30 UTC
(In reply to comment #2)
> Christ, I don't use web browser to send data to apache httpd. How do I 
display
> the  httpd header? Can you tell me a command how to do that, turn on the 
debug
> or something?
> The version of mod_proxy_ajp is Apache/2.2.3
> The version of mod_jk is I don't know how to see them?

Since it sounds like your configuration is pretty much just out of the box, 
then edit $CATALINA_HOME/conf/logging.properties and add a line that looks 
like:

org.apache.jk.level = FINE

Since we only care about the Content-Length, doing something like:

LogFormat "%h %l %u %t \"%r\" %>s %b %{content-length}i"

in httpd.conf should be enough.
Comment 6 william.barker 2007-06-12 19:12:30 UTC
(In reply to comment #3)
> The version I have checked here is 1.2.18. You should upgrade to the latest
> version of mod_jk (currently 1.2.23) if possible.

Can you actually read all of the request body?  It shouldn't throw an 
exception with a 3GB request body while parsing the header, and you should get 
dropped connections (at least if you aren't using the latest from SVN), but it 
looks to me like mod_jk will truncate the input (unless sizeof(int) >= 16 on 
your platform).
Comment 7 Mark Thomas 2007-06-17 14:35:43 UTC
Created attachment 20361 [details]
Rough patch for http 2.2.x only

mod_proxy_ajp is now fine with POSTs > 2GB
mod_jk fails to send any content for POSTs > 2GB

I have attached a patch that fixes mod_jk for me but:
- it only fixes this for httpd 2.2.x
- I haven't looked at any of the other servers
- I don't trust my c coding skills enough to commit it

Hopefully, one of the mod_jk folks will pick this up and turn it into a
reasonable patch.
Comment 8 Mark Thomas 2007-06-17 14:36:24 UTC
Changing component as testing shows mod_proxy_ajp now works but mod_jk still fails.
Comment 9 william.barker 2007-06-17 15:04:50 UTC
(In reply to comment #7)
> Created an attachment (id=20361) [edit]
> Rough patch for http 2.2.x only
> mod_proxy_ajp is now fine with POSTs > 2GB
> mod_jk fails to send any content for POSTs > 2GB
> I have attached a patch that fixes mod_jk for me but:
> - it only fixes this for httpd 2.2.x
> - I haven't looked at any of the other servers
> - I don't trust my c coding skills enough to commit it
> Hopefully, one of the mod_jk folks will pick this up and turn it into a
> reasonable patch.


It won't compile for httpd 1.3.x or IIS.  But it looks like a 
s/apr_off_t/jk_uint64_t/g should turn into a resonable patch.

Quibbles:  
1) The jk_ajp12_worker.c is no longer supported, so that one can be dropped 
(but it is harmless to include it).
2) There look like there are a couple of tab-to-space problems in the patch.
Comment 10 Rainer Jung 2007-06-17 21:26:59 UTC
I'm also in it, but didn't yet test my patch. It will use jk_uint64_t anf the
apache parts are not very far away from your patch.

I will commit after my tests, but then we will still need to test the Windows
and IIS+Netscape parts.
Comment 11 Rainer Jung 2007-06-19 09:45:21 UTC
I committed a patch for mod_jk. A dev-tarball of mod_jk can be found at

  http://people.apache.org/~rjung/mod_jk-dev/HugeContent/

This needs to be tested well, especially in combination with Bill Barker's
patches for the Tomcat connectors.

Please report any test results here.
Comment 12 Mark Thomas 2007-06-19 19:55:40 UTC
I have a simple test that submits 4G data and counts the bytes received.

Tested with:
Tomcat 5.5.HEAD, Win XP SP2 Home, using Java APR connector.

Apache 1.3.37, mod_jk 1.2.24-dev built locally with httpd on the same box as
Tomcat fails with a 400 - Invalid content length. The error appears to be httpd
generated.

Apache 2.2.4, mod_jk 1.2.24-dev built locally with httpd on the same box as
Tomcat works (all 4G data is received)

IIS 5, Win2k running on VMWare, mod_jk 1.2.24-dev built locally is *very* slow
for smaller files and fails completely (connection is dropped) for 4G files.
Comment 13 Rainer Jung 2007-06-20 01:35:53 UTC
Mark, thanks a lot.

I think the failing with httpd 1.3 is not that bad. It's very outdated now and
support for huge content could be handled as yet another reason to update. From
the mod_jk perspective everything is analogous, but it relies an the web server
being able to accept such a large value in the content length header.

Concerning IIS: The IIS special part of the patch is close to being non
existant. All the relevant parts are the same as for IIS. So I strongly assume,
that the slowlyness hasn't been introduced by this fix. Maybe it's a VMWare problem.

I hope Mladen will retest IIS basic performance to double check.

Thanks again for the test!

I'll let this BZ open and we can close it in a couple of days, if no negative
side effects come up.
Comment 14 Rainer Jung 2007-06-20 11:30:11 UTC
Now I did some tests as well. I tested downloads and not uploads. Three test cases:

a) size a little below 2^31 Bytes
b) size a little above 2^31 Bytes
c) size a little above 2^32 Bytes

Platform Solaris 10 Sparc, Tomcat 5.5.24 candidate using Java APR connector,
mod_jk 1.2.24-dev.

All Apache builds 32 Bit binaries.

Apache 1.3.37: a: OK, b: OK, but "0" as size in access log, c: OK, but size mod
2^ 32 in access log

Apache 2.0.59: a: OK, b: OK, but negative size in access log, c: OK, but size
mod 2^ 32 in access log

Apache 2.2.4: a,b,c: OK, correct sizes in access log

But: as soon as the size goes over 2^31, I don't get a Content-Length header
back. I needed to fake a little: when I use static content Tomcat tries to load
it into memory, so I used a little servlet to produce the download content on
the fly. But the servlet API call to set the content length header only allows
an int as the argument. I instead used the setHeader method for Content-Length.
This works well below 2^31, but above the Tomcat AJP connector does not handle
the header back to mod_jk. Nevertheless wget could retrieve the complete
message, and I *think* Tomcat transparently switched to transfer encoding chunked.
Comment 15 Rainer Jung 2007-06-20 11:58:32 UTC
Same test on same system with Sun Webserver 6.1 SP 7 64Bit: a,b,c are OK with
correct length in access log, but again the Content-Length header seems not to
come back from Tomcat when size is bigger than 2^31.

Download speed was the same for Sun, Apache 1.3, 2.0 and 2.2 (12MB/s; of course
this is not a performance test, it's just one huge download on a slow machine).
Using the direct http connector of Tomcat resulted in app. 15% faster download.

The absoulte number is not important (the servlet was doing cracy stuff, so
generating the content was CPU bound).

The Tomcat access log by the way, shows correct length below 2^31, it shows a
"-" for case b and the size mod 2^32 for case c.