What version the stacktrace is from?

> at java.nio.Buffer.limit(Buffer.java:275)

The line matches Java 8u31 sources.
The IAE is triggered by the following check:

[[[
public final Buffer limit(int newLimit) {
   if ((newLimit > capacity) || (newLimit < 0))
            throw new IllegalArgumentException();
]]]

> org.apache.coyote.ajp.AjpNioProcessor.readSocket(AjpNioProcessor.java:179)

The above does not match Tomcat 7 sources. By method name I guess that is line 356.

[[[
    private int readSocket(byte[] buf, int pos, int n, boolean block)
            throws IOException {
        int nRead = 0;
        ByteBuffer readBuffer =
                socketWrapper.getSocket().getBufHandler().getReadBuffer();
        readBuffer.clear();
        readBuffer.limit(n);
]]]

Apparently it tries to read n bytes from the Socket read buffer at once. The buffer is smaller that that thus the IAE.

(In reply to Konstantin Kolinko from comment #1)
> What version the stacktrace is from?
> 
> > at java.nio.Buffer.limit(Buffer.java:275)
> 
> The line matches Java 8u31 sources.
> The IAE is triggered by the following check:
> 
> [[[
> public final Buffer limit(int newLimit) {
>    if ((newLimit > capacity) || (newLimit < 0))
>             throw new IllegalArgumentException();
> ]]]
> 
> > org.apache.coyote.ajp.AjpNioProcessor.readSocket(AjpNioProcessor.java:179)
> 
> The above does not match Tomcat 7 sources. By method name I guess that is
> line 356.

The reference in comment #1 says Tomcat 8.0.17. I'm sorry I didn't set the correct version. I was able to reproduce this in 7.0.56, 7.0.57, 7.0.50, 8.0.17 and 8.0.20 with varying stack traces. The one posted here appears to be from Tomcat 8.0.17.

> [[[
>     private int readSocket(byte[] buf, int pos, int n, boolean block)
>             throws IOException {
>         int nRead = 0;
>         ByteBuffer readBuffer =
>                 socketWrapper.getSocket().getBufHandler().getReadBuffer();
>         readBuffer.clear();
>         readBuffer.limit(n);
> ]]]
> 
> Apparently it tries to read n bytes from the Socket read buffer at once. The
> buffer is smaller that that thus the IAE.

That seems like a reasonable diagnosis, but I'm not familiar enough with how these components interact to understand the root cause and most appropriate fix.

I suspect this is a rare problem since most people probably stick to the default packetSize (8192) with the AJP connector.

I have a test case for this. Sending any AJP request body chunk message with size > socket read buffer is sufficient to trigger this with NIO or NIO2.

BIO, APR and 9.0.x are all unaffected.

The fix looks to be simple. I need to clean everything up and should be able to commit a fix shortly.

Fixed in 8.0.x and 8.0.21 onwards and in 7.0.x for 7.0.60 onwards.

Neither trunk nor 6.0x. were affected.

(In reply to Mark Thomas from comment #4)
> Fixed in 8.0.x and 8.0.21 onwards and in 7.0.x for 7.0.60 onwards.


Excellent, I'll reproduce, then update and re-test in my environment.

> Neither trunk nor 6.0x. were affected.

That was my expectation based upon Rémy'y comments about the refactoring in trunk having unified lots of things; this bug would have been much more obvious with other connectors.

I just re-built with Tomcat 7.0.x trunk and I'm now getting a different exception:

SEVERE: Servlet.service() for servlet velocity threw exception
java.nio.BufferOverflowException
        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:189)
        at org.apache.coyote.ajp.AjpNioProcessor.output(AjpNioProcessor.java:305)
        at org.apache.coyote.ajp.AbstractAjpProcessor$SocketOutputBuffer.doWrite(AbstractAjpProcessor.java:1234)
        at org.apache.coyote.Response.doWrite(Response.java:499)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:402)
        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:480)
        at org.apache.catalina.connector.OutputBuffer.realWriteChars(OutputBuffer.java:485)
        at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:464)
        at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java:302)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:527)
        at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java:152)
        at org.apache.velocity.io.VelocityWriter.flushBuffer(VelocityWriter.java:129)
        at org.apache.velocity.io.VelocityWriter.write(VelocityWriter.java:306)
        at org.apache.velocity.io.VelocityWriter.write(VelocityWriter.java:322)
        at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:491)
        at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
        at org.apache.velocity.Template.merge(Template.java:356)
        at org.apache.velocity.Template.merge(Template.java:260)
        at org.apache.velocity.tools.view.VelocityView.performMerge(VelocityView.java:942)
        at org.apache.velocity.tools.view.VelocityView.merge(VelocityView.java:902)
        at org.apache.velocity.tools.view.VelocityViewServlet.mergeTemplate(VelocityViewServlet.java:318)
        at org.apache.velocity.tools.view.VelocityLayoutServlet.mergeTemplate(VelocityLayoutServlet.java:247)
        at org.apache.velocity.tools.view.VelocityViewServlet.doRequest(VelocityViewServlet.java:220)
        at org.apache.velocity.tools.view.VelocityViewServlet.doGet(VelocityViewServlet.java:182)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
  [...]

Might this be the other side of the buffer coin?

I added some debug logging to the line immediately before the writeBuffer.put(), and I got these logs before the exception:

Mar 07, 2015 2:02:09 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=65536, starting at offset=0, len=79

Mar 07, 2015 2:02:09 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=6, starting at offset=0, len=6

Mar 07, 2015 2:02:09 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=65536, starting at offset=0, len=113

Mar 07, 2015 2:02:09 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=6, starting at offset=0, len=6

Mar 07, 2015 2:02:10 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=65536, starting at offset=0, len=42

Mar 07, 2015 2:02:10 PM org.apache.coyote.ajp.AjpNioProcessor output
WARNING: Writing to output buffer of capacity=8192, position=0, limit=8192, remaining=8192 from source buffer of size=65536, starting at offset=0, len=8200

That is a different bug. It is in output rather than input. Please open a new issue.

Filed the output-oriented bug under bug #57674.