Bug 56449 - WebSocket session get closed if in onOpen will sendText (sync mode)
WebSocket session get closed if in onOpen will sendText (sync mode)
Status: RESOLVED FIXED
Product: Tomcat 8
Classification: Unclassified
Component: WebSocket
8.0.5
PC Linux
: P2 major (vote)
: ----
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2014-04-23 16:13 UTC by vlcvlc
Modified: 2014-04-24 12:12 UTC (History)
0 users



Attachments
The exception captured (1.30 KB, text/plain)
2014-04-23 16:13 UTC, vlcvlc
Details

Note You need to log in before you can comment on or make changes to this bug.
Description vlcvlc 2014-04-23 16:13:18 UTC
Created attachment 31552 [details]
The exception captured

@ServerEndpoint("/push/echo")
public class EchoAnnotation {    
    @OnOpen
    public void open(Session session) {
        try { 
            if(session.isOpen()) session.getBasicRemote().sendText(
            "Welcome!");
        } catch(Exception e) { 
            try {
                session.close();
            } catch (IOException e1) {
                // Ignore
            }
        }
    }


Doing session.getAsynRemote() is working.
Important: This errors happens with thousands of websockets.
Comment 1 Mark Thomas 2014-04-23 21:45:31 UTC
At the moment, I don't see a code path that could trigger this. The session is open when it is created so something must be closing it very quickly. My assumption is that there must be an error path triggering this but I don;t see it so far. What would be very useful is the stack trace from when the session is closed. If I add some debug logging are you able to build Tomcat from source and run your test to collect some more information?
Comment 2 Konstantin Preißer 2014-04-23 22:16:29 UTC
Hi Mark,

FYI, I can reproduce the given stack trace if I use a client that sends an initial Websocket Request but immediately closes the TCP connection. E.g., when running the following Java code on Windows while Tomcat (trunk) is running using the NIO connector:

        String str = "GET /examples/websocket/chat HTTP/1.1\r\n"
                + "Host: localhost:8080\r\n"
                + "Sec-WebSocket-Version: 13\r\n"
                + "Origin: http://localhost:8080\r\n"
                + "Sec-WebSocket-Key: 8Ydn5qfZ/eg0v37Jh4n6tw==\r\n"
                + "Connection: keep-alive, Upgrade\r\n"
                + "Upgrade: websocket\r\n\r\n";
        byte[] bytes = str.getBytes("UTF-8");

        try (Socket s = new Socket("127.0.0.1", 8080)) {
        	s.getOutputStream().write(bytes);
        }

I then get the following exceptions (I think the first is expected):

24-Apr-2014 00:07:57.351 SEVERE [http-nio-8080-exec-1] websocket.chat.ChatAnnotation.onError Chat Error: java.io.IOException: java.util.concurrent.ExecutionException: java.nio.channels.ClosedChannelException
 java.io.IOException: java.util.concurrent.ExecutionException: java.nio.channels.ClosedChannelException
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessageBlock(WsRemoteEndpointImplBase.java:242)
	at org.apache.tomcat.websocket.WsSession.sendCloseMessage(WsSession.java:486)
	at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:417)
	at org.apache.tomcat.websocket.WsSession.close(WsSession.java:394)
	at org.apache.tomcat.websocket.WsSession.close(WsSession.java:388)
	at websocket.chat.ChatAnnotation.broadcast(ChatAnnotation.java:99)
	at websocket.chat.ChatAnnotation.start(ChatAnnotation.java:59)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.tomcat.websocket.pojo.PojoEndpointBase.doOnOpen(PojoEndpointBase.java:58)
	at org.apache.tomcat.websocket.pojo.PojoEndpointServer.onOpen(PojoEndpointServer.java:70)
	at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler.init(WsHttpUpgradeHandler.java:129)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:678)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1575)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1533)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.nio.channels.ClosedChannelException
	at org.apache.tomcat.websocket.FutureToSendHandler.get(FutureToSendHandler.java:102)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessageBlock(WsRemoteEndpointImplBase.java:235)
	... 20 more
Caused by: java.nio.channels.ClosedChannelException
	at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:265)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:474)
	at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:122)
	at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:184)
	at org.apache.coyote.http11.upgrade.NioServletOutputStream.doWriteInternal(NioServletOutputStream.java:94)
	at org.apache.coyote.http11.upgrade.NioServletOutputStream.doWrite(NioServletOutputStream.java:61)
	at org.apache.coyote.http11.upgrade.AbstractServletOutputStream.writeInternal(AbstractServletOutputStream.java:158)
	at org.apache.coyote.http11.upgrade.AbstractServletOutputStream.write(AbstractServletOutputStream.java:124)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible(WsRemoteEndpointImplServer.java:99)
	at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:81)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:390)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessage(WsRemoteEndpointImplBase.java:284)
	at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessageBlock(WsRemoteEndpointImplBase.java:230)
	... 20 more


24-Apr-2014 00:07:57.397 SEVERE [http-nio-8080-exec-1] org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process Error reading request, ignored
 java.lang.IllegalStateException: The WebSocket session [0] has been closed and no method (apart from close()) may be called on a closed session
	at org.apache.tomcat.websocket.WsSession.checkState(WsSession.java:652)
	at org.apache.tomcat.websocket.WsSession.addMessageHandler(WsSession.java:168)
	at org.apache.tomcat.websocket.pojo.PojoEndpointBase.doOnOpen(PojoEndpointBase.java:81)
	at org.apache.tomcat.websocket.pojo.PojoEndpointServer.onOpen(PojoEndpointServer.java:70)
	at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler.init(WsHttpUpgradeHandler.java:129)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:678)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1575)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1533)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)



This happens only if the client immediately closes the connection after sending the request. If it waits a bit before closing it (or I just connect with a browser like Firefox), I don't get any error.
However, I don't know if the cause of that exception is the same as in the original report.
Comment 3 Mark Thomas 2014-04-24 06:06:57 UTC
Thanks for that.

An immediate close was one of the options I considered but I dismissed it as I didn't think the close message could have been read that early. What i didn't consider was that the close of the connection rather than the close message may have triggered the state change on the session. I'll dig some more.
Comment 4 vlcvlc 2014-04-24 08:08:56 UTC
(In reply to Mark Thomas from comment #1)
> At the moment, I don't see a code path that could trigger this. The session
> is open when it is created so something must be closing it very quickly. My
> assumption is that there must be an error path triggering this but I don;t
> see it so far. What would be very useful is the stack trace from when the
> session is closed. If I add some debug logging are you able to build Tomcat
> from source and run your test to collect some more information?

Mark, HI

I am evaluating Tomcat websockets implementation. For this, I use a client side websockets library that creates connection after connection. After a while (<10000 connections) the error is thrown and Tomcat 8 server hangs for a while. 

I attached HTTP NIO connector configuration used:
    <Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
               connectionTimeout="20000" keepAliveTimeout="-1" redirectPort="8443" 
	       acceptCount="20000" acceptorThreadCount="8" 
	       minSpareThreads="100" maxThreads="10000" 
	       maxConnections="-1" URIEncoding="UTF-8" 
               maxKeepAliveRequests="-1" />


Regards, 
Cristian
Comment 5 vlcvlc 2014-04-24 08:11:50 UTC
(In reply to Mark Thomas from comment #3)
> Thanks for that.
> 
> An immediate close was one of the options I considered but I dismissed it as
> I didn't think the close message could have been read that early. What i
> didn't consider was that the close of the connection rather than the close
> message may have triggered the state change on the session. I'll dig some
> more.

FYI, NO errors reproduced in other Web/EE java servers

Regards,
Cristian
Comment 6 Mark Thomas 2014-04-24 08:14:29 UTC
(In reply to Mark Thomas from comment #3)
> What I
> didn't consider was that the close of the connection rather than the close
> message may have triggered the state change on the session.

Neither did I consider that the onOpen method could close the session. That appears to be what is triggering this particular error.

This has been fixed in 8.0.x for 8.0.6 onwards and 7.0.x for 7.0.54 onwards.
Comment 7 vlcvlc 2014-04-24 12:12:39 UTC
(In reply to Mark Thomas from comment #6)
> (In reply to Mark Thomas from comment #3)
> > What I
> > didn't consider was that the close of the connection rather than the close
> > message may have triggered the state change on the session.
> 
> Neither did I consider that the onOpen method could close the session. That
> appears to be what is triggering this particular error.
> 
> This has been fixed in 8.0.x for 8.0.6 onwards and 7.0.x for 7.0.54 onwards.

I will come back when will test the release 8.0.6 
Thank you for the fastness. 


Regards, 
Cristian