Bug 50394 - InternalAprInputBuffer.fill() doesn't deal correctly with EOF
Summary: InternalAprInputBuffer.fill() doesn't deal correctly with EOF
Alias: None
Product: Tomcat Native
Classification: Unclassified
Component: Library (show other bugs)
Version: 1.1.20
Hardware: PC Linux
: P2 normal with 10 votes (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2010-12-01 11:38 UTC by Hugh Warrington
Modified: 2011-09-16 18:52 UTC (History)
1 user (show)

2011-08-26_tc55_50394_InternalAprInputBuffer.patch (834 bytes, patch)
2011-08-26 12:33 UTC, Konstantin Kolinko
Details | Diff
2011-08-26_tc6_50394_InternalAprInputBuffer.patch (768 bytes, patch)
2011-08-26 12:34 UTC, Konstantin Kolinko
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hugh Warrington 2010-12-01 11:38:03 UTC
I'm using tomcat-native-1.1.20 and tomcat-6.0.29 on CentOS 5.5, and I've written a servlet that is sent large (gigabyte) streams of bytes over HTTP.

From time to time it fails with the following stacktrace:

Caused by: java.io.IOException
        at org.apache.coyote.http11.InternalAprInputBuffer.fill(InternalAprInputBuffer.java:798)
        at org.apache.coyote.http11.InternalAprInputBuffer$SocketInputBuffer.doRead(InternalAprInputBuffer.java:827)
        at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
        at org.apache.coyote.http11.InternalAprInputBuffer.doRead(InternalAprInputBuffer.java:738)
        at org.apache.coyote.Request.doRead(Request.java:428)
        at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)
        at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:403)
        at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)
        at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at java.io.FilterInputStream.read(FilterInputStream.java:107)

Unfortunately, InternalAprInputBuffer throws away the error code on line 798, but I've run it under a debugger and nRead was -70014.

If you look in the source of apr-1.3.8 at include/apr_errno.h, you'll see this is -APR_EOF.

The bug is in the implementation of recvbb at line 892 onwards in tomcat-native-1.1.20-src/jni/native/src/network.c. Specifically, at the end of the function we check for APR_SUCCESS, and assume all other codes are an error, returning -ss. It should also check for EOF (APR_STATUS_IS_EOF), and return zero. (Or at least, InternalAprInputBuffer assumes that a return code of 0 <=> EOF).

Incidentally, apr-1.3.8/include/apr_network_io.h says in its comment on apr_socket_recv() that "It is possible for both bytes to be received and an APR_EOF or other error to be returned.". This is a lie. All provided implementations of apr_socket_recv return with *len == 0 in case of APR_EOF.
Comment 1 jfclere 2011-03-10 07:43:44 UTC
The code in org/apache/coyote/http11/InternalAprInputBuffer is:
            nRead = Socket.recvbb(socket, 0, buf.length - lastValid);
            if (nRead > 0) {
                bbuf.get(buf, pos, nRead);
                lastValid = pos + nRead;
            } else {
                if ((-nRead) == Status.ETIMEDOUT || (-nRead) == Status.TIMEUP) {
                    throw new SocketTimeoutException(sm.getString("iib.failedread"));
                } else {
                    throw new IOException(sm.getString("iib.failedread"));
So returning 0 won't help.
Comment 2 Hugh Warrington 2011-03-15 11:52:12 UTC
Ok, well perhaps my proposed fix won't work, but this is still an issue since it's not conforming to the InputStream specification.
Comment 3 jfclere 2011-07-19 09:39:49 UTC
Fix by 1148254 backport proposed.
Comment 4 Rohini 2011-08-25 16:58:33 UTC
We hit this issue yesterday with tomcat-native-1.1.20 when uploading 1G file. But found that if we did not use BufferedInputStream and read directly into a buffer it worked fine. 

InputStream in = request.getInputStream(); //new BufferedInputStream(request.getInputStream());
int bytesRead;
      while ((bytesRead = in.read(buf)) != -1) {
        out.write(buf, 0, bytesRead);
Comment 5 Konstantin Kolinko 2011-08-25 17:26:42 UTC
(In reply to comment #3)
> Fix by 1148254 backport proposed.

There is some confusion on the status of this issue.
The r1148254 mentioned above was essentially reverted by r1148815.

The actual fix is in Tomcat-Native 1.1.22 by r1148814.

Native 1.1.22 is already released - you may download it.

It seems that part of r1148815 still needs a backport to 6.0 and 5.5:
s/} else {/} else if (nRead != 0) {/
Comment 6 Konstantin Kolinko 2011-08-26 12:33:39 UTC
Created attachment 27437 [details]
Comment 7 Konstantin Kolinko 2011-08-26 12:34:10 UTC
Created attachment 27438 [details]
Comment 8 Konstantin Kolinko 2011-08-26 12:43:33 UTC
Proposed the patches for Tomcat 6.0 and 5.5.
Comment 9 Mark Thomas 2011-09-02 11:53:16 UTC
Fixed in 6.0.x and will be included in 6.0.34 onwards
Comment 10 Konstantin Kolinko 2011-09-16 18:52:09 UTC
Fixed in 5.5.x with r1171684 and will be in 5.5.34 onwards.