Bug 53406 - Stack overflow in connector
Stack overflow in connector
Status: RESOLVED FIXED
Product: Tomcat 7
Classification: Unclassified
Component: Connectors
7.0.27
PC All
: P2 normal (vote)
: ---
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2012-06-13 03:51 UTC by Filip Hanik
Modified: 2012-06-13 15:48 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Filip Hanik 2012-06-13 03:51:40 UTC
found against trunk on Jun 18, 2012

SEVERE:
java.lang.StackOverflowError
        at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:128)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:223)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:110)
        at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:209)
        at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:124)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:569)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
Comment 1 Konstantin Kolinko 2012-06-13 07:08:16 UTC
The following are remarkable points in this stacktrace:

> org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
> org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)

NioEndpoint$Poller.add(NioChannel socket, int interestOps):
[[[
            if (close) {
                processSocket(socket, SocketStatus.STOP, false);
            }
]]]

> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)

[[[
            } else {
                //...
                socket.getSocket().getPoller().add(socket.getSocket());
]]]

so this happens with an application that uses Comet (those event() calls), when longPoll() is processed, but endpoint is being closed at the same time.

It looks like SocketStatus.STOP value is not being honored. The poller#close flag is inaccessible from outside.
Comment 2 Filip Hanik 2012-06-13 15:35:38 UTC
Thanks for the analysis.

Here is what I think will fix it

Index: java/org/apache/coyote/http11/Http11NioProcessor.java
===================================================================
--- java/org/apache/coyote/http11/Http11NioProcessor.java       (revision 1349898)
+++ java/org/apache/coyote/http11/Http11NioProcessor.java       (working copy)
@@ -155,7 +155,7 @@

         rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);

-        if (error) {
+        if (error || status==SocketStatus.STOP) {
             return SocketState.CLOSED;
         } else if (!comet) {
             if (keepAlive) {

What this does, is that it prevents reuse of connections after a Comet transaction has been notified of a STOP event.
Comment 3 Mark Thomas 2012-06-13 15:45:11 UTC
I was thinking along the same lines but fixing it in the Adaptor rather than just for NIO. The issue with my fix is that it somewhat abused the error flag (by changing it's meaning to mean "close the connection for any reason" rather than "something went wrong".

Your fix is clearer as to intention but it needs to be made to APR too. I'll so that now unless you beat me to it.
Comment 4 Mark Thomas 2012-06-13 15:48:45 UTC
Fixed for trunk and 7.0.x and will be included in 7.0.28 onwards.