Bug 60437

Summary: SSL Handshake implementation in AsycnChannelWrapperSecure does not handle SSLEngineResult.Status.BUFFER_OVERFLOW state
Product: Tomcat 8 Reporter: Dharmesh Patel <dpatel>
Component: WebSocketAssignee: Tomcat Developers Mailing List <dev>
Severity: normal    
Priority: P2    
Version: 8.0.38   
Target Milestone: ----   
Hardware: PC   
OS: All   
Attachments: sample program

Description Dharmesh Patel 2016-12-01 12:08:46 UTC
Created attachment 34494 [details]
sample program

This defect is related to an issue reported previously (https://bz.apache.org/bugzilla/show_bug.cgi?id=54997) .
With the above fix, although BUFFER_UNDERFLOW is being handled, there is still an issue with BUFFER_OVERFLOW. 

Issue is reproduced by using WebSocketContainer to connect to a server which requires clientcert authentication (  SSLEnabled="true" clientAuth="true" )

In this case, during the SSL handshake, when AsyncChannelWrapperSecure gets a handshakeStatus of NEED_UNWRAP, it needs to dynamically resize the DUMMY buffer, before invoking sslEngine.unwrap(socketReadBuffer, DUMMY), as recommended by http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.html . 

The handshake fails with the following error :
javax.websocket.DeploymentException: The HTTP request to initiate the WebSocket connection failed
	at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:423)
	at com.wss.WSSCLientUsingTomcatWSContainer.main(WSSCLientUsingTomcatWSContainer.java:27)
Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLException: TODO
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WrapperFuture.get(AsyncChannelWrapperSecure.java:508)
	at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:367)
	... 1 more
Caused by: javax.net.ssl.SSLException: TODO
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.checkResult(AsyncChannelWrapperSecure.java:427)
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.run(AsyncChannelWrapperSecure.java:391)
Comment 1 Remy Maucherat 2016-12-01 13:08:40 UTC
Please don't add developers individually, thanks.
That DUMMY buffer is 8KB. This is probably not right and it could be either sslEngine.getSession().getPacketBufferSize() or 16921. I don't see why dynamic resize would be needed.
Comment 2 Dharmesh Patel 2016-12-01 13:12:38 UTC
Yes, resizing  to sslEngine.getSession().getPacketBufferSize() or 16921 would work, since socketReadBuffer is set similarly.
Comment 3 Remy Maucherat 2016-12-01 15:10:22 UTC
I'll try the simplest solution first then, the default buffer size will be 16921. The change will be in 9M14, 8.5.9, 8.0.40 and 7.0.74.