Bug 60451 - java.lang.ArrayIndexOutOfBoundsException when a servlet writes more than the output buffer max length on a connection to be upgraded to HTTP/2
Summary: java.lang.ArrayIndexOutOfBoundsException when a servlet writes more than the ...
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 8.5.8
Hardware: PC Linux
: P2 regression (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-12-07 16:05 UTC by Ludovic Pénet
Modified: 2016-12-08 22:24 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ludovic Pénet 2016-12-07 16:05:04 UTC
I have a simple servlet producing files in Excel format using Apache POI.

It basically does a 

    wbk.write(resp.getOutputStream());

where wbk is an instance of org.apache.poi.ss.usermodel.Workbook and resp is an instance of HttpServletResponse.

When my SSL connector is parametered this way

    <Connector port="8443" SSLEnabled="true"
                protocol="org.apache.coyote.http11.Http11AprProtocol"
                maxThreads="150" scheme="https" secure="true"
                sslProtocol="TLS"
                sslImplementationName="org.apache.tomcat.util.net.openssl.OpenSSLImplementation"
		SSLCertificateFile="${catalina.home}/conf/certificate.crt"
                SSLCertificateKeyFile="${catalina.home}/conf/privateKey.key"
		server="Apache-Coyote/1.1"
                URIEncoding="UTF-8">
      <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
    </Connector>

in server.xml, I have the following exception.

07-Dec-2016 16:24:58.603 GRAVE [https-openssl-apr-8443-exec-19] org.apache.catalina.core.StandardWrapperValve.invoke "Servlet.service()" pour la servlet fr.senat.presences.servlets.ExcelPresencesServlet a généré une exception
 java.lang.ArrayIndexOutOfBoundsException: -23
	at org.apache.coyote.http2.HPackHuffman.encode(HPackHuffman.java:441)
	at org.apache.coyote.http2.HpackEncoder.writeHuffmanEncodableValue(HpackEncoder.java:228)
	at org.apache.coyote.http2.HpackEncoder.encode(HpackEncoder.java:190)
	at org.apache.coyote.http2.Http2UpgradeHandler.writeHeaders(Http2UpgradeHandler.java:534)
	at org.apache.coyote.http2.Stream.writeHeaders(Stream.java:326)
	at org.apache.coyote.http2.StreamProcessor.prepareResponse(StreamProcessor.java:98)
	at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:263)
	at org.apache.coyote.Response.action(Response.java:170)
	at org.apache.coyote.Response.sendHeaders(Response.java:352)
	at org.apache.coyote.http2.Stream$StreamOutputBuffer.doWrite(Stream.java:582)
	at org.apache.coyote.Response.doWrite(Response.java:517)
	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:351)
	at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:808)
	at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:713)
	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:391)
	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:369)
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96)
	at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
	at org.apache.poi.poifs.storage.BigBlock.doWriteData(BigBlock.java:67)
	at org.apache.poi.poifs.storage.DocumentBlock.writeData(DocumentBlock.java:195)
	at org.apache.poi.poifs.storage.BigBlock.writeBlocks(BigBlock.java:98)
	at org.apache.poi.poifs.storage.DocumentBlock.writeBlocks(DocumentBlock.java:34)
	at org.apache.poi.poifs.filesystem.POIFSDocument$BigBlockStore.writeBlocks(POIFSDocument.java:547)
	at org.apache.poi.poifs.filesystem.POIFSDocument.writeBlocks(POIFSDocument.java:303)
	at org.apache.poi.poifs.filesystem.POIFSFileSystem.writeFilesystem(POIFSFileSystem.java:380)
	at org.apache.poi.hssf.usermodel.HSSFWorkbook.write(HSSFWorkbook.java:1308)
	at fr.senat.exporters.ExcelExporter.exportActivite(ExcelExporter.java:578)
	at fr.senat.presences.servlets.ExcelPresencesServlet.doGetActivite(ExcelPresencesServlet.java:89)
	at fr.senat.presences.servlets.ExcelPresencesServlet.doGet(ExcelPresencesServlet.java:39)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
	at fr.senat.presences.filters.PseudoRequestScopeEMFilter.doFilter(PseudoRequestScopeEMFilter.java:95)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
	at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:589)
	at fr.senat.tomcat.valve.JwtValve.handleAuthentication(JwtValve.java:320)
	at fr.senat.tomcat.valve.JwtValve.invoke(JwtValve.java:235)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
	at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:291)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
	at org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:219)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.http2.StreamProcessor.run(StreamProcessor.java:63)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)


When debugging, I noticed that POI tries to write more than 8192 bytes to the output buffer. 8192 being the output buffer size, it is not resized to be bigger than 8192 bytes. This case is properly handled when not using HTTP/2.
Comment 1 Ludovic Pénet 2016-12-07 17:21:12 UTC
Well, my first analysis of this problem was wrong.

After further debugging, it appears that the problem is rather in the "Content-Disposition" header value.

As we are in France, it sometimes contains non ascii chars. In this case, char é caused the exception in HPackHuffman.encode.

So, I changed the way I set the header from :

        resp.setHeader("Content-Disposition", "attachment;filename=\"" + filename + "\"");

to :

        URLEncoder enc = new URLEncoder();
        resp.setHeader("Content-Disposition", "attachment; filename*=UTF-8''" + enc.encode(filename, "UTF-8"));

and it works.
Comment 2 Michael Osipov 2016-12-08 10:59:03 UTC
This one is worth reading: http://stackoverflow.com/a/30446122/696632
Comment 3 Ludovic Pénet 2016-12-08 11:14:42 UTC
Agreed. I left the bug opened because the exception raised was quite unclear to me and having another error trace would be great.
Comment 4 Mark Thomas 2016-12-08 22:24:50 UTC
Neither the HTTP/2 spec nor the HPACK spec define the encoding to be used to convert characters to bytes for header values once you step outside of ASCII so to some extent this is going to be a lottery.

Tomcat's implementation was meant to use the unicode code point but failed to take account of the fact the byte is signed in Java. I've fixed this and improved the error message if you try to send a header containing a character with a code point above 255. I also added some test cases.

As an aside, your original example should now work.

Fixed in:
- trunk for 9.0.0.M15 onwards
- 8.5.x for 8.5.10 onwards