Bug 60437 - SSL Handshake implementation in AsycnChannelWrapperSecure does not handle SSLEngineResult.Status.BUFFER_OVERFLOW state
Summary: SSL Handshake implementation in AsycnChannelWrapperSecure does not handle SSL...
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: WebSocket (show other bugs)
Version: 8.0.38
Hardware: PC All
: P2 normal (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2016-12-01 12:08 UTC by Dharmesh Patel
Modified: 2016-12-01 15:10 UTC (History)
0 users

sample program (1.43 KB, text/plain)
2016-12-01 12:08 UTC, Dharmesh Patel

Note You need to log in before you can comment on or make changes to this bug.
Description Dharmesh Patel 2016-12-01 12:08:46 UTC
Created attachment 34494 [details]
sample program

This defect is related to an issue reported previously (https://bz.apache.org/bugzilla/show_bug.cgi?id=54997) .
With the above fix, although BUFFER_UNDERFLOW is being handled, there is still an issue with BUFFER_OVERFLOW. 

Issue is reproduced by using WebSocketContainer to connect to a server which requires clientcert authentication (  SSLEnabled="true" clientAuth="true" )

In this case, during the SSL handshake, when AsyncChannelWrapperSecure gets a handshakeStatus of NEED_UNWRAP, it needs to dynamically resize the DUMMY buffer, before invoking sslEngine.unwrap(socketReadBuffer, DUMMY), as recommended by http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLEngine.html . 

The handshake fails with the following error :
javax.websocket.DeploymentException: The HTTP request to initiate the WebSocket connection failed
	at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:423)
	at com.wss.WSSCLientUsingTomcatWSContainer.main(WSSCLientUsingTomcatWSContainer.java:27)
Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLException: TODO
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WrapperFuture.get(AsyncChannelWrapperSecure.java:508)
	at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:367)
	... 1 more
Caused by: javax.net.ssl.SSLException: TODO
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.checkResult(AsyncChannelWrapperSecure.java:427)
	at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.run(AsyncChannelWrapperSecure.java:391)
Comment 1 Remy Maucherat 2016-12-01 13:08:40 UTC
Please don't add developers individually, thanks.
That DUMMY buffer is 8KB. This is probably not right and it could be either sslEngine.getSession().getPacketBufferSize() or 16921. I don't see why dynamic resize would be needed.
Comment 2 Dharmesh Patel 2016-12-01 13:12:38 UTC
Yes, resizing  to sslEngine.getSession().getPacketBufferSize() or 16921 would work, since socketReadBuffer is set similarly.
Comment 3 Remy Maucherat 2016-12-01 15:10:22 UTC
I'll try the simplest solution first then, the default buffer size will be 16921. The change will be in 9M14, 8.5.9, 8.0.40 and 7.0.74.