We had a customer that could not log into our web application from their browser. The problem occurred during the full handshake between the 2 APR connectors running in 2 separate webapp services in Tomcat: the webui client socket (port 8445) and our rest api server socket (port 8443). The error below indicates there was a malloc issue while parsing the OCSP entry in the AuthorityInfoAccess extension. The catalina.out file contained this “Handshake failed” message: Apr 08, 2019 10:59:04 PM org.apache.tomcat.util.net.AprEndpoint setSocketOptions FINE: Handshake failed: error:27072041:OCSP routines:OCSP_parse_url:malloc failure There did not appear to be a malloc issue. There was plenty of memory available when I ran top when this problem occurred. The few tcpdump dumps we have all show that the server port (8443) does a FIN/ACK followed by a RST after it issues a Server Hello Done and receives the client’s certificate. There is no alert in the tcpdump. Note that the log level of this error message is FINE (DEBUG), but for a handshake failure, I would expect a log level of SEVERE (ERROR). I contacted SafeLogic which handles our openssl package. They said it appears that Tomcat Native retrieves the OCSP url from the Authority Information Access X509 extension using its own parsing routines. It then calls the OpenSSL function OCSP_parse_url with the resulting url. The certificate did not contain an OCSP access entry in its Authority Information Access (AIA) extension. It only contained a CA-Issuers access entry. SafeLogic suspected that for this case Tomcat is passing a NULL url to OCSP_parse_url(), which would result in the above openssl error. The AIA entry in the certificate is as follows: Authority Information Access: CA Issuers – URI:http://wxyz-dc-01.wxyz.local/pki/WXYZ-DC-01.WXYZ.local_IssuingCA.crt We compiled libtcnative with OCSP disabled and the customer was able to log into the system in question. Software Versions: ------------------------ Tomcat 8.5.30 libtcnative-1_1.2.16F with APR FIPS-140 support openssl 1.0.2n Java 1.8.0_131 Ubuntu 16.04
Fortunately, there is only a single code path where OCSP_parse_url is found, so this ought to be easy to track down. I can't see a code path where a NULL OCSP URL would be passed-into OCSP_parse_url, but I could see where a zero-length string might be passed-in. Are you able to build from source, and can I give you a patch to get some more debugging information?
Yes, we untar the source and then build like this: Untar the source tar -xvf tomcat-native_1.2.16.orig.tar.xz tar -xvf tomcat-native_1.2.16-1build1.debian.tar.xz Install APR sudo apt-get install libapr1-dev cd tomcat-native-1.2.16-src/native ./configure --with-java-home=/usr/lib/jvm/java-8-openjdk-amd64 --with-ssl=yes make
Created attachment 36532 [details] Patch to avoid empty URLs It wasn't clear what the best way was to log debug information other than just to stdout, so I'm including a one-liner patch that will hopefully avoid attempts to parse a useless OCSP URL. Let me know if it is useful to you.
I installed the libtcnative files which included the patch you sent (and compiled with ENABLE_OCSP=1) on one of our system that contained a certificate with the Authority Information Access X509 extension problem. The handshake was successful and I was able to log into our webui using the patch. Thank you for your help.
Re-opening so we can track getting the fix committed to the code base.
This bug was the same as https://github.com/apache/tomcat-native/pull/4 I used the PR to fix this. The fix will be in 1.2.22 onwards.