Bug 53406 - Stack overflow in connector
Summary: Stack overflow in connector
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Connectors (show other bugs)
Version: 7.0.27
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-13 03:51 UTC by Filip Hanik
Modified: 2012-06-13 15:48 UTC (History)
0 users


Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Filip Hanik 2012-06-13 03:51:40 UTC 
found against trunk on Jun 18, 2012

SEVERE:
java.lang.StackOverflowError
        at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:128)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:223)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.valves.ValveBase.event(ValveBase.java:204)
        at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:110)
        at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:209)
        at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:124)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:569)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:999)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:596)
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
Comment 1 Konstantin Kolinko 2012-06-13 07:08:16 UTC 
The following are remarkable points in this stacktrace:

> org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:730)
> org.apache.tomcat.util.net.NioEndpoint$Poller.add(NioEndpoint.java:1008)

NioEndpoint$Poller.add(NioChannel socket, int interestOps):
[[[
            if (close) {
                processSocket(socket, SocketStatus.STOP, false);
            }
]]]

> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.longPoll(Http11NioProtocol.java:277)

[[[
            } else {
                //...
                socket.getSocket().getPoller().add(socket.getSocket());
]]]

so this happens with an application that uses Comet (those event() calls), when longPoll() is processed, but endpoint is being closed at the same time.

It looks like SocketStatus.STOP value is not being honored. The poller#close flag is inaccessible from outside.
Comment 2 Filip Hanik 2012-06-13 15:35:38 UTC 
Thanks for the analysis.

Here is what I think will fix it

Index: java/org/apache/coyote/http11/Http11NioProcessor.java
===================================================================
--- java/org/apache/coyote/http11/Http11NioProcessor.java       (revision 1349898)
+++ java/org/apache/coyote/http11/Http11NioProcessor.java       (working copy)
@@ -155,7 +155,7 @@

         rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);

-        if (error) {
+        if (error || status==SocketStatus.STOP) {
             return SocketState.CLOSED;
         } else if (!comet) {
             if (keepAlive) {

What this does, is that it prevents reuse of connections after a Comet transaction has been notified of a STOP event.
Comment 3 Mark Thomas 2012-06-13 15:45:11 UTC 
I was thinking along the same lines but fixing it in the Adaptor rather than just for NIO. The issue with my fix is that it somewhat abused the error flag (by changing it's meaning to mean "close the connection for any reason" rather than "something went wrong".

Your fix is clearer as to intention but it needs to be made to APR too. I'll so that now unless you beat me to it.
Comment 4 Mark Thomas 2012-06-13 15:48:45 UTC 
Fixed for trunk and 7.0.x and will be included in 7.0.28 onwards.