Bug 45516 - All Tomcat http threads stuck in java.net.SocketOutputStream.socketWrite0(Native Method)
Summary: All Tomcat http threads stuck in java.net.SocketOutputStream.socketWrite0(Nat...
Status: RESOLVED INVALID
Alias: None
Product: Tomcat 6
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 6.0.14
Hardware: Other Linux
: P2 normal (vote)
Target Milestone: default
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-07-31 05:28 UTC by Ville Hartikainen
Modified: 2008-08-01 07:47 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ville Hartikainen 2008-07-31 05:28:14 UTC
Between random intervals our website running with Liferay + Tomcat 6.0.14 gets stuck. 

CPU utilization is close to nothing. After getting a thread dump from the running java process, a following stacktraces are seen in all of the http-threads:

Is anybody able to tell, whether this is a bug in Tomcat, JVM, in our configuration or have I just diagnosed the problem incorrectly? 

Something to workaround this problem?
- - -
"http-9095-28" daemon prio=1 tid=0x0000000044743190 nid=0x3bc runnable [0x000000005124a000..0x000000005124be10]
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:737)
	at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:434)
	at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
	at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:761)
	at org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:118)
	at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:570)
	at org.apache.coyote.Response.doWrite(Response.java:560)
	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
	at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:354)
	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:381)
	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:370)
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
	- locked <0x00002aaaf85f8ac0> (a java.io.BufferedOutputStream)
	at com.liferay.util.servlet.ServletResponseUtil.write(ServletResponseUtil.java:194)
	at com.liferay.filters.strip.StripFilter.doFilter(StripFilter.java:249)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.filters.secure.SecureFilter.doFilter(SecureFilter.java:144)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.filters.compression.CompressionFilter.doFilter(CompressionFilter.java:122)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.filters.doubleclick.DoubleClickFilter.doFilter(DoubleClickFilter.java:135)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.portal.servlet.filters.layoutcache.LayoutCacheFilter.doFilter(LayoutCacheFilter.java:167)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.portal.servlet.filters.autologin.AutoLoginFilter.doFilter(AutoLoginFilter.java:106)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at com.liferay.portal.servlet.filters.virtualhost.VirtualHostFilter.doFilter(VirtualHostFilter.java:147)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:654)
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:445)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:292)
	at com.liferay.portal.servlet.filters.virtualhost.VirtualHostFilter.doFilter(VirtualHostFilter.java:197)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
	at java.lang.Thread.run(Thread.java:595)
Comment 1 Ville Hartikainen 2008-08-01 00:52:56 UTC
Version information of used JDK and Tomcat, as reported by "catalina.sh version"
------------------
Using JRE_HOME:       /usr/java/jdk1.5.0_13/
Server version: Apache Tomcat/6.0.14
Server built:   Jul 20 2007 04:17:30
Server number:  6.0.14.0
OS Name:        Linux
OS Version:     2.6.18-53.1.4.el5
Architecture:   amd64
JVM Version:    1.5.0_13-b05
JVM Vendor:     Sun Microsystems Inc.
Comment 2 Mark Thomas 2008-08-01 07:47:17 UTC
This occasionally gets reported on the users list. The most recent occurrence appears to be this thread:
http://markmail.org/message/hghfhwoxdoa6fg5s

My best guess at the moment as to the root cause is some form of network oddity.

The users list is the best place to get help with this issue as bugzilla is really suited to being used as a support forum.

If you investigations identify a bug in Tomcat, feel free to re-open this issue and provide the appropriate information.