Bug 56239 - NullPointerException when finishing response under the Linux traffic control
Summary: NullPointerException when finishing response under the Linux traffic control
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Connectors (show other bugs)
Version: 7.0.42
Hardware: PC Linux
: P2 major (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-03-10 07:35 UTC by Lsk
Modified: 2018-10-22 03:20 UTC (History)
0 users



Attachments
stacktrace (9.09 KB, text/plain)
2014-03-11 07:51 UTC, Lsk
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Lsk 2014-03-10 07:35:47 UTC
hi, when i using apache benchmark to test my tomcat instance, i got a NullPointerException 

this is stacktrace:

[2013-12-21 19:47:00][ERROR] [http-nio-5000-exec-6]2013-12-21 19:47:00: org.apache.coyote.http11.AbstractHttp11Processor.endRequest Error finishing response
 java.lang.NullPointerException
        at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:210)
        at org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:202)
        at org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:765)
        at org.apache.coyote.Response.action(Response.java:171)
        at org.apache.coyote.http11.AbstractOutputBuffer.endRequest(AbstractOutputBuffer.java:302)
        at org.apache.coyote.http11.InternalNioOutputBuffer.endRequest(InternalNioOutputBuffer.java:120)
        at org.apache.coyote.http11.AbstractHttp11Processor.endRequest(AbstractHttp11Processor.java:1660)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1053)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1686)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

[2013-12-21 19:46:58][ERROR] [http-nio-5000-exec-5]2013-12-21 19:46:58: org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process Error reading request, ignored
 java.lang.NullPointerException
        at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.release(Http11NioProtocol.java:221)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:616)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1686)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

I am using "org.apache.coyote.http11.Http11NioProtocol" in server.xml like this:
<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" ... />

I have already test 2 cases: 
1) without Linux Traffic Control, my testcase was very ok!
2) when i using linux tc to control the output network traffic. and tomcat gave me a NullPointerException above. that the tomcat instance still at running state,(some requests has failed)
I test many times with tc, NullPointerException almost has been happened 
I was changed tomcat version at 7.0.52 with the same problem
I have seen 7.0.39-7.0.52's changelog, and find nothing with something like this Exception. 

so please help ..
Comment 1 Mark Thomas 2014-03-10 10:40:08 UTC
No information on the configuration of traffic control.

No information on the load test being performed.

No information in the application being tested.

In short, no way anyone investigating this bug could possibly begin to try and reproduce it.

Please provide the above information and the errors (with stack traces) obtained when running on the latest stable 7.0.x release.
Comment 2 Lsk 2014-03-11 07:51:26 UTC
Created attachment 31380 [details]
stacktrace
Comment 3 Lsk 2014-03-11 07:53:47 UTC
hi Thomas, 

here is my tc configuration: 

[root@aceZtomcatZ59099c87Ze182Z4f13Z ~]# tc qdisc show dev eth1: 
qdisc htb 1: root refcnt 2 r2q 10 default 10 direct_packets_stat 0

[root@aceZtomcatZ59099c87Ze182Z4f13Z ~]# tc class show dev eth1
class htb 1:505 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:10 parent 1:1 prio 0 rate 10486Kbit ceil 94372Kbit burst 1597b cburst 1580b 
class htb 1:504 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:507 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:506 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:500 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:503 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:502 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:509 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:508 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:1 root rate 104858Kbit ceil 104858Kbit burst 1585b cburst 1585b 
class htb 1:510 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 
class htb 1:511 parent 1:1 prio 0 rate 3146Kbit ceil 94372Kbit burst 1599b cburst 1580b 

I am running tomcat under "admin" account, uid is 500

[root@aceZtomcatZ59099c87Ze182Z4f13Z ~]# id -u admin
500

Performance Test command, on another machine :
[root@aceZtestZ607c2b5bZ0bcbZ4768ZbZ ~]# ab -c 100 -n 50000 http://121.199.24.74:9527/tmall.com_files/T1oKF9Fo8gXXcR3qY4-200-320.png_q90.jpg
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 121.199.24.74 (be patient)
apr_socket_recv: Connection reset by peer (104)
Total of 3031 requests completed


the attarchment is stacktrace info.

thanks a lot !
Comment 4 Mark Thomas 2014-03-11 09:44:25 UTC
1. Fix your broken web application - you should not be bundling multiple versions of the Servlet API in your web application.

2. Please provide a stack trace from the latest stable 7.0.x release. 7.0.39 in not the latest stable 7.0.x release.
Comment 5 Lsk 2014-03-12 06:34:09 UTC
Well, I have fixed the second problme that you say (switch tomcat version to last_release_stable) and everything is okay ...


i have been look around the all tomcat's changelog from 7.0.39 to 7.0.52 at this page: http://tomcat.apache.org/tomcat-7.0-doc/changelog.html  
but found nothing about with "NIO Connector maybe thrown a NullPointerException in some case" 

what is the difference between this two version 7.0.39-7.0.52 ?
Comment 6 Mark Thomas 2014-03-12 12:54:46 UTC
In the original description you stated that you changed Tomcat version to 7.0.52 and still observed the same problem.

In comment #5 you stated that you could not reproduce the problem when using 7.0.52.

Which of those statement is correct?
Comment 7 Mark Thomas 2014-03-12 13:03:47 UTC
It is not immediately obvious that this is the same issue so yes, please create a new bugzilla issue and provide the necessary information to enable this to be reproduced.
Comment 8 Mark Thomas 2014-03-12 13:04:05 UTC
(In reply to Mark Thomas from comment #7)
> It is not immediately obvious that this is the same issue so yes, please
> create a new bugzilla issue and provide the necessary information to enable
> this to be reproduced.

Ignore this comment. Wrong bug.
Comment 9 Lsk 2014-03-12 16:15:53 UTC
(In reply to Mark Thomas from comment #6)
> In the original description you stated that you changed Tomcat version to
> 7.0.52 and still observed the same problem.
> 
> In comment #5 you stated that you could not reproduce the problem when using
> 7.0.52.
> 
> Which of those statement is correct?

I mean i changed tomcat version to 7.0.52 then, NullPointerException has gone ..
and my testcases was right..

but i don't understand what is the difference between 7.0.39 and 7.0.52 ?
Comment 10 Christopher Schultz 2014-03-12 18:18:52 UTC
You can read the changelog (as you have done) or you can view diffs in Subversion. Nobody is going to hunt-down the exact patch that fixes this particular issue for you.

As for the changelog, search for anything NIO-related. A lot of things have been modified in the connectors recently to make them more similar to each other and easier to maintain. It's possible that your edge-case was removed as a part of that effort, and so it doesn't appear as a particular bugfix, etc.

If you really want to track down exactly what fixed it, start with 7.0.40 and keep upgrading until you hit a release that does not reproduce the problem. Then you can look at diffs.