Bug 64426 - TestSSLHostConfigCompat crashes with LibreSSL 2.9.0
Summary: TestSSLHostConfigCompat crashes with LibreSSL 2.9.0
Alias: None
Product: Tomcat Native
Classification: Unclassified
Component: Library (show other bugs)
Version: 1.2.23
Hardware: All FreeBSD
: P2 major (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2020-05-12 09:30 UTC by Michael Osipov
Modified: 2020-08-20 14:14 UTC (History)
0 users

Test output (540.68 KB, text/plain)
2020-05-12 09:30 UTC, Michael Osipov
Sample test result (540 bytes, text/plain)
2020-05-12 09:31 UTC, Michael Osipov
Crash file 1 (35.81 KB, text/plain)
2020-05-12 09:31 UTC, Michael Osipov
Crash file 2 (36.08 KB, text/plain)
2020-05-12 09:31 UTC, Michael Osipov
Crash file 3 (36.08 KB, text/plain)
2020-05-12 09:32 UTC, Michael Osipov

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Osipov 2020-05-12 09:30:14 UTC
How to reproduce:

Obtain master (51f949dc6e0b6e4e27972b8ba2d0a2626fc3c1c5) for tomcat-native, revert b8649e81458194d70667952d9e26df82a79c773f. Obtain Tomcat master.

* Apply this patch against tcnative:
diff --git a/native/src/ssl.c b/native/src/ssl.c
index fcb2a9ec..ad0a17fd 100644
--- a/native/src/ssl.c
+++ b/native/src/ssl.c
@@ -1328 +1328,3 @@ TCN_IMPLEMENT_CALL(jlong /* SSL * */, SSL, newSSL)(TCN_STDARGS,
-        tcn_ThrowException(e, "cannot create new ssl");
+        char err[256];
+        ERR_error_string(SSL_ERR_get(), err);
+        tcn_Throw(e, "Cannot create new SSL (%s)", err);

* Compile tomcat-native against LibreSSL 2.9.0 and 2.9.1
Comipling previous versions of LibreSSL is tricky:
** Clone from GitHub
** Switch to v2.9.0 or v2.9.1
** Apply patch:
diff --git a/update.sh b/update.sh
index 3315311..bc291e2 100755
--- a/update.sh
+++ b/update.sh
@@ -18 +18 @@ fi
- git pull --rebase)
+ )
** Modify OPENBSD_BRANCH to libressl-v2.9.0 or libressl-v2.9.1
** Perform ./update.sh
*** NOTE: When switching between versions all produce needs to be wiped out, along with openbsd/! 
* Prefixes: /tmp/tcnative, /tmp/libressl-2.9.0, /tmp/libressl-2.9.1
* Run "ant test -Dtest.entry=org.apache.tomcat.util.net.TestSSLHostConfigCompat" with both. 2.9.1 shall succeed, 2.9.0 will fail with two issues ("java.lang.Exception: Unable to use Java keystores with LibreSSL" can be ignored, this is expected):
**  java.lang.Exception: Cannot create new SSL (error:14FFF0C3:SSL routines:(UNKNOWN)SSL_internal:null ssl ctx)
**     [junit] # JRE version: OpenJDK Runtime Environment (8.0_252-b09) (build 1.8.0_252-b09)
    [junit] # Java VM: OpenJDK 64-Bit Server VM (25.252-b09 mixed mode bsd-amd64 compressed oops)
    [junit] # Problematic frame:
    [junit] # C  [libssl.so.47+0x3f2e9]  SSL_CTX_use_certificate+0x9
    [junit] #

Log files attached.

I don't know whether it is our code or a bug in LibreSSL below 2.9.1. We might say, since 2.9.1 is the first stable release from the 2.9.x, it is a minimum version.

All tests performed with:
* Compiler:
FreeBSD clang version 9.0.1 (git@github.com:llvm/llvm-project.git c1a0a213378a458fbea1a5c77b315c7dce08fd05) (based on LLVM 9.0.1)
Target: x86_64-unknown-freebsd12.1
Thread model: posix
InstalledDir: /usr/bin
* Java:
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (The FreeBSD Project)(build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (The FreeBSD Project)(build 25.252-b09, mixed mode)
* OS: FreeBSD 12.1-STABLE #1 r360428
Comment 1 Michael Osipov 2020-05-12 09:30:41 UTC
Created attachment 37231 [details]
Test output
Comment 2 Michael Osipov 2020-05-12 09:31:11 UTC
Created attachment 37232 [details]
Sample test result
Comment 3 Michael Osipov 2020-05-12 09:31:32 UTC
Created attachment 37233 [details]
Crash file 1
Comment 4 Michael Osipov 2020-05-12 09:31:45 UTC
Created attachment 37234 [details]
Crash file 2
Comment 5 Michael Osipov 2020-05-12 09:32:00 UTC
Created attachment 37235 [details]
Crash file 3
Comment 6 Christopher Schultz 2020-05-12 20:13:03 UTC
Thanks for diving-into this. I've wanted to support LibreSSL for a long time, but I'm not really an expert in the arcana required to work with OpenSSL.
Comment 7 Michael Osipov 2020-05-12 21:13:45 UTC
I have already identified other compat issues with LibreSSL which can be easily fixed with the appropriate ifdefs.

Getting back to this, my lack of strong C knowledge and the associated Java code is too little to make any progress, but I can happily provide more testing if necessary.
Comment 8 Mark Thomas 2020-08-20 14:14:36 UTC
As far as I can tell the root cause is a LibreSSL bug. I see the same behaviour with the current Tomcat Native source.

I have added some debugging trace to LibreSSL 2.9.0 it it shows a crash at the point where the internal ssl_set_cert(CERT *c, X509 *x) method is called. That looks like memory corruption to me.