Bug 55602 - JRE crashes during websocket communication
Summary: JRE crashes during websocket communication
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Connectors (show other bugs)
Version: trunk
Hardware: PC Linux
: P2 major (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2013-09-27 14:27 UTC by Stefan Thurnherr
Modified: 2013-10-03 14:05 UTC (History)
0 users


Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Thurnherr 2013-09-27 14:27:19 UTC
During some initial tests with the Websocket implementation in Tomcat 7.0.42 the jre crashed seemingly randomly with the following crash log.

We got the same crash 3 times during a full work day while testing an extended version of the chat examples/ app in message (not binary) mode with a few clients. No idea how to reproduce though as the crashes appeared seemingly randomly. The chat had always been working fine for some time before the crash occurred.

Using APR based Apache Tomcat Native library 1.1.27 using APR version 1.4.6.

# A fatal error has been detected by the Java Runtime Environment:
#  SIGSEGV (0xb) at pc=0x00007f4b6a72c9dd, pid=10100, tid=139961597916928
# JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 1.7.0_40-b43)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libtcnative-1.so.0.1.27+0x129dd]  Java_org_apache_tomcat_jni_Socket_send+0x15d
# Core dump written. Default location: /home/asok/myself/core or core.10100
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

---------------  T H R E A D  ---------------

Current thread (0x00007f4928010000):  JavaThread "http-apr-10088-exec-7" daemon [_thread_in_native, id=18750, stack(0x00007f4b59437000,0x00007f4b59538000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000040

RAX=0x0000000000000000, RBX=0x00007f49280011d0, RCX=0x0000000000000000, RDX=0x00007f4b59534128
RSP=0x00007f4b59534110, RBP=0x00007f49280101e8, RSI=0x00007f4b59534130, RDI=0x0000000000000000
R8 =0x00007f4b59534130, R9 =0x0000000000000001, R10=0x00007f4b8d01852d, R11=0x00007f4b985e3ed0
R12=0x0000000000000000, R13=0x0000000000000000, R14=0x00007f4b59536210, R15=0x00007f4928010000
RIP=0x00007f4b6a72c9dd, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004

Top of Stack: (sp=0x00007f4b59534110)
0x00007f4b59534110:   327473657547202a 6f6a207361682031
0x00007f4b59534120:   0000002e64656e69 0000000000000001
0x00007f4b59534130:   0000000000000081 0000000000000000
0x00007f4b59534140:   0000000000000000 0000000000000008
0x00007f4b59534150:   0000000000000000 0000000000000000
0x00007f4b59534160:   0000000000000000 0000000000000000
0x00007f4b59534170:   0000000000000000 0000000000000000
0x00007f4b59534180:   0000000000003528 0000000000000000
0x00007f4b59534190:   0000000000005978 0000000000000000
0x00007f4b595341a0:   0000000000005980 0000000000000000
0x00007f4b595341b0:   0000000000000008 0000000000000000
0x00007f4b595341c0:   0000000000000000 0000000000000000
0x00007f4b595341d0:   0000000000000000 0000000000000000
0x00007f4b595341e0:   0000000000000000 0000000000000000
0x00007f4b595341f0:   0000000000000000 0000000000000000
0x00007f4b59534200:   0000000000000000 0000000000000000
0x00007f4b59534210:   0000000000000000 00007f4b00000008
0x00007f4b59534220:   0000000000000000 00007f4b00000000
0x00007f4b59534230:   00007f4900000000 00007f4900000000
0x00007f4b59534240:   00007f4b00000000 00007f4b00000000
0x00007f4b59534250:   0000000000000000 0000000000000000
0x00007f4b59534260:   0000000000000000 0000000000000000
0x00007f4b59534270:   0000000000000000 0000000000000000
0x00007f4b59534280:   0000000000000000 0000000000000000
0x00007f4b59534290:   0000000000000000 0000000000000000
0x00007f4b595342a0:   0000000000000000 0000000000000000
0x00007f4b595342b0:   0000000000000000 0000000000000000
0x00007f4b595342c0:   0000000000000000 0000000000000b00
0x00007f4b595342d0:   0000000000000000 0000000000000000
0x00007f4b595342e0:   0000000000000000 0000000000000000
0x00007f4b595342f0:   0000000000000000 0000000000000000
0x00007f4b59534300:   0000000000000000 0000000000000000 

Instructions: (pc=0x00007f4b6a72c9dd)
0x00007f4b6a72c9bd:   89 ea 4c 89 f6 48 89 ef ff 90 40 06 00 00 48 8b
0x00007f4b6a72c9cd:   43 30 48 8b 7b 18 48 8d 54 24 18 48 8d 74 24 20
0x00007f4b6a72c9dd:   ff 50 40 89 c3 e9 47 ff ff ff be 58 00 00 00 48
0x00007f4b6a72c9ed:   89 ef e8 ec a1 ff ff b8 a8 ff ff ff e9 6a ff ff 

Register to memory mapping:

RAX=0x0000000000000000 is an unknown value
RBX=0x00007f49280011d0 is an unknown value
RCX=0x0000000000000000 is an unknown value
RDX=0x00007f4b59534128 is pointing into the stack for thread: 0x00007f4928010000
RSP=0x00007f4b59534110 is pointing into the stack for thread: 0x00007f4928010000
RBP=0x00007f49280101e8 is an unknown value
RSI=0x00007f4b59534130 is pointing into the stack for thread: 0x00007f4928010000
RDI=0x0000000000000000 is an unknown value
R8 =0x00007f4b59534130 is pointing into the stack for thread: 0x00007f4928010000
R9 =0x0000000000000001 is an unknown value
R10=0x00007f4b8d01852d is at code_begin+1197 in an Interpreter codelet
method entry point (kind = native)  [0x00007f4b8d018080, 0x00007f4b8d018e80]  3584 bytes
R11=0x00007f4b985e3ed0: <offset 0x183ed0> in /lib/x86_64-linux-gnu/libc.so.6 at 0x00007f4b98460000
R12=0x0000000000000000 is an unknown value
R13=0x0000000000000000 is an unknown value
R14=0x00007f4b59536210 is pointing into the stack for thread: 0x00007f4928010000
R15=0x00007f4928010000 is a thread

Stack: [0x00007f4b59437000,0x00007f4b59538000],  sp=0x00007f4b59534110,  free space=1012k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libtcnative-1.so.0.1.27+0x129dd]  Java_org_apache_tomcat_jni_Socket_send+0x15d

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.apache.tomcat.jni.Socket.send(J[BII)I+0
j  org.apache.coyote.http11.upgrade.UpgradeAprProcessor.write(I)V+14
j  org.apache.coyote.http11.upgrade.UpgradeOutbound.write(I)V+5
j  org.apache.catalina.websocket.WsOutbound.doWriteBytes(Ljava/nio/ByteBuffer;Z)V+68
j  org.apache.catalina.websocket.WsOutbound.doWriteText(Ljava/nio/CharBuffer;Z)V+65
j  org.apache.catalina.websocket.WsOutbound.writeTextMessage(Ljava/nio/CharBuffer;)V+44
j  com.company.app.handler.log.chat.ChatWebSocketServlet.broadcast(Ljava/lang/String;)V+66
j  com.company.app.handler.log.chat.ChatWebSocketServlet$ChatMessageInbound.onOpen(Lorg/apache/catalina/websocket/WsOutbound;)V+46
j  org.apache.catalina.websocket.StreamInbound.onUpgradeComplete()V+22
j  org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+224
j  org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run()V+167
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
Comment 1 Christopher Schultz 2013-09-27 14:54:31 UTC
Possible dupe of bug #51813

Can you re-try with tcnative 1.1.28? (I mis-typed in your other bug that 1.1.27 is latest... it's 1.1.28 though it does not appear to be on the download page for some reason). Try looking here:
Comment 2 Stefan Thurnherr 2013-09-27 15:05:27 UTC
(The other bug #55588 is not created by me, I just subscribed as the jre crash looked similar. But it does not seem to have to do with websockets. Our tomcat is very stable when not using websockets.)
Comment 3 Konstantin Preißer 2013-09-27 15:14:41 UTC

I remember that I got crashes when using a WebSocket application on Tomcat 7.0.27 with TC Native 1.1.23 and Java 1.7.0_03 on Windows Server 2008 x86 - see [1].

I do not know if it is related, but the Stack traces look similar.

However, since then, I never have done additional testing with Tomcat 7, Websockets and APR connector.

[1] http://markmail.org/message/5m5qmtu3g67zgia3
Comment 4 Mark Thomas 2013-09-30 19:40:17 UTC
This has been fixed in 8.0.x and 7.0.x and will be included in 8.0.0-RC4 onwards and 7.0.46 onwards.
Comment 5 Stefan Thurnherr 2013-10-03 13:56:33 UTC
(In reply to Mark Thomas from comment #4)
> This has been fixed in 8.0.x and 7.0.x and will be included in 8.0.0-RC4
> onwards and 7.0.46 onwards.
Do you mean 7.0.46, or rather 7.0.43? Current stable release is 7.0.42, and 7.0.42 is also the release where I observed this bug.
Comment 6 Chuck Caldarale 2013-10-03 14:01:15 UTC
(In reply to st.mailinglists from comment #5)
> Do you mean 7.0.46, or rather 7.0.43? Current stable release is 7.0.42, and
> 7.0.42 is also the release where I observed this bug.

7.0.46, since the attempted releases for 7.0.43 through 7.0.45 were canceled.
Comment 7 Stefan Thurnherr 2013-10-03 14:05:55 UTC
(In reply to Chuck Caldarale from comment #6)
> (In reply to st.mailinglists from comment #5)
> > Do you mean 7.0.46, or rather 7.0.43? Current stable release is 7.0.42, and
> > 7.0.42 is also the release where I observed this bug.
> 7.0.46, since the attempted releases for 7.0.43 through 7.0.45 were canceled.
Great, thanks for the clarification!