Unfortunately I don't have any actual repro steps for this issue but in our test environment we've seen NullPointerExceptions like the one shown below, and we've also from our logs concluded that calls to HttpServletRequest.getServletPath() at the same time returned null (which according to the Javadoc should never happen as far as I understand). The issue is not easily reproducible - we've just seen it a few times in our load tests so I speculate that it may be due to some kind of race condition. java.lang.NullPointerException at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at com.REDACTED.SecurityFilter.processRequest(SecurityFilter.java:335) at com.REDACTED.SecurityFilter.doFilter(SecurityFilter.java:176) at com.REDACTED.AbstractFilter.doFilter(AbstractFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) We're using Spring 4.3.3 if that's relevant (which I doubt it is). We've recently upgraded from Tomcat 8.0.33 to 8.0.37 and we didn't see the issue before (but given that the issue isn't easily reproduced that may of course just have been luck).
All the evidence points to an application bug rather than a Tomcat bug. It appears that the application is retaining a reference to an object (probably a request) and attempting to use it after Tomcat has completed the processing for that request. The NPE is triggered because the FilterChain has been recycled. The ServletPath is null because the request has been recycled. This recycling occurs once the request has been processed.
Thank you for your swift reply Mark! It makes sense I guess so I'm closing this. However, looking more at my logs I found that several things suggested that processing of at least parts of the request was made on a different thread (and if Tomcat finishes the processing of the request on that thread that would indeed mean that we have the situation you describe once control is returned to the original request thread) - so Spring MVC, which have support for asynchronous handling (although we don't use it) seemed like a likely culprit (we've recently updated Spring as well). But then I found the stack trace below, where my entire filter chain (which is pretty long) is missing. Doesn't this suggest some kind of Tomcat issue? Other requests for the same resource are passed through the filter chain (without Tomcat restarting in between or anything like that). The problem just went away after 40 minutes or so. java.lang.NullPointerException at com.REDACTED.ManifestController.getHeaderConfig(ManifestController.java:209) at sun.reflect.GeneratedMethodAccessor474.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:114) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:528) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1100) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:687) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)
(In reply to Jens Borgland from comment #2) <snip/> > But then I found the stack > trace below, where my entire filter chain (which is pretty long) is missing. > Doesn't this suggest some kind of Tomcat issue? Unlikely. The root cause is likely to be a reference being retained to the request object and that object being recycled after the filterchain is set up but before it is used. You might find it helpful to set this system property to true: org.apache.catalina.connector.RECYCLE_FACADES Bugzilla is not well suited to support. The users list is the place to follow this up.