64419 – Build problem with tomcat-native on FreeBSD 11.3/LibreSSL

Bug 64419 - Build problem with tomcat-native on FreeBSD 11.3/LibreSSL

Summary: Build problem with tomcat-native on FreeBSD 11.3/LibreSSL

Status:	RESOLVED FIXED

Alias:	None

Product:	Tomcat Native
Classification:	Unclassified
Component:	Library (show other bugs)
Version:	1.2.23
Hardware:	PC FreeBSD

Importance:	P2 regression (vote)
Target Milestone:	---
Assignee:	Tomcat Developers Mailing List

URL:
Keywords:

Depends on:
Blocks:

Reported:	2020-05-10 22:25 UTC by gessel
Modified:	2020-05-13 12:34 UTC (History)
CC List:	3 users (show)

Attachments
work directory of failed build (408.33 KB, application/octet-stream) 2020-05-10 22:30 UTC, gessel	Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description gessel 2020-05-10 22:25:58 UTC

--- src/ssl.lo ---
In file included from src/ssl.c:24:
./include/ssl_private.h:220:9: warning: 'OPENSSL_VERSION' macro redefined [-Wmacro-redefined]
#define OPENSSL_VERSION                  SSLEAY_VERSION
        ^
/usr/local/include/openssl/crypto.h:329:9: note: previous definition is here
#define OPENSSL_VERSION         0
        ^
src/ssl.c:301:9: warning: implicit declaration of function 'SSL_CTX_set_keylog_callback' is invalid in C99 [-Wimplicit-function-declaration]
        SSL_CTX_set_keylog_callback(ctx, ssl_keylog_callback);
        ^
src/ssl.c:789:41: error: use of undeclared identifier 'thread_exit_key'; did you mean 'pthread_exit'?
    err = apr_threadkey_private_create(&thread_exit_key, _ssl_thread_exit,
                                        ^~~~~~~~~~~~~~~
                                        pthread_exit
/usr/include/pthread.h:215:7: note: 'pthread_exit' declared here
void            pthread_exit(void *) __dead2;
                ^
src/ssl.c:789:58: error: use of undeclared identifier '_ssl_thread_exit'
    err = apr_threadkey_private_create(&thread_exit_key, _ssl_thread_exit,
                                                         ^
src/ssl.c:796:5: error: use of undeclared identifier 'threadkey_initialized'
    threadkey_initialized = 1;
    ^
src/ssl.c:799:5: warning: implicit declaration of function 'ssl_thread_setup' is invalid in C99 [-Wimplicit-function-declaration]
    ssl_thread_setup(tcn_global_pool);
    ^
3 warnings and 3 errors generated.
*** [src/ssl.lo] Error code 1

FreeBSD 11.3-RELEASE-p8 #0 r360490


1.2.23 compiled successfully, but 1.2.24 is throwing errors.

tomcat-native-1.2.23               <   needs updating (index has 1.2.24)

Comment 1 gessel 2020-05-10 22:30:41 UTC

Created attachment 37223 [details]
work directory of failed build

.7z file - tar.gz was over 1MB

Comment 2 Michael Osipov 2020-05-11 09:59:43 UTC

As previously discussed with gessel and the Port maintainer of libtcnative, the code works with OpenSSL. We don't have an official position no LibreSSL.

I will look into this later this day.

Comment 3 Michael Osipov 2020-05-11 12:05:49 UTC

OK, there are several issues here. They aren't FreeBSD specific, but the issue is the incomplete LibreSSL coverage:

> ./include/ssl_private.h:220:9: warning: 'OPENSSL_VERSION' macro redefined [-Wmacro-redefined]
> #define OPENSSL_VERSION                  SSLEAY_VERSION
>         ^
> /usr/local/include/openssl/crypto.h:329:9: note: previous definition is here
> #define OPENSSL_VERSION         0
>         ^

The ifdef around this block assumes that LibreSSL at some point did not have these definitions.

> src/ssl.c:301:9: warning: implicit declaration of function 'SSL_CTX_set_keylog_callback' is invalid in C99
>       [-Wimplicit-function-declaration]
>         SSL_CTX_set_keylog_callback(ctx, ssl_keylog_callback);
>         ^

LibeSSL does not support this. This patch solves the issue:
> diff --git a/native/include/ssl_private.h b/native/include/ssl_private.h
> index d88e393d..26495e46 100644
> --- a/native/include/ssl_private.h
> +++ b/native/include/ssl_private.h
> @@ -241,7 +241,7 @@
>  #define TLS_server_method                SSLv23_server_method
>  #endif /* OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER) */
> 
> -#if OPENSSL_VERSION_NUMBER >= 0x10101000L
> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L && !defined(LIBRESSL_VERSION_NUMBER)
>  #define HAVE_KEYLOG_CALLBACK
>  #endif

> src/ssl.c:789:41: error: use of undeclared identifier 'thread_exit_key'; did you mean 'pthread_exit'?
>     err = apr_threadkey_private_create(&thread_exit_key, _ssl_thread_exit,
>                                         ^~~~~~~~~~~~~~~
>                                         pthread_exit
> /usr/include/pthread.h:215:7: note: 'pthread_exit' declared here
> void            pthread_exit(void *) __dead2;
>                 ^
> src/ssl.c:789:58: error: use of undeclared identifier '_ssl_thread_exit'
>     err = apr_threadkey_private_create(&thread_exit_key, _ssl_thread_exit,
>                                                          ^
> src/ssl.c:796:5: error: use of undeclared identifier 'threadkey_initialized'
>     threadkey_initialized = 1;
>     ^
> src/ssl.c:799:5: warning: implicit declaration of function 'ssl_thread_setup' is invalid in C99
>       [-Wimplicit-function-declaration]
>     ssl_thread_setup(tcn_global_pool);
>     ^

As sad as it seems. With the ifdefs around threaded init and OpenSSL 1.1.0+ the coverage of LibreSSL seems to be incomplete because LibreSSL reports OpenSSL version 0.

To solve this properly, we need to do the following:

* Require a minimum LibreSSL
* Test for that LibreSSL version in ./configure
* Figure out whether LibreSSL inits threading itself like OpenSSL 1.1.0+
* Revise code blocks for LibreSSL compat:
> [mosipov@mika-ion ~/Projekte/tomcat-native/native]$ grep -r "OPENSSL_VERSION_NUMBER < 0x10100000L" .
> ./include/ssl_private.h:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./include/ssl_private.h:#endif /* OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER) */
> ./include/ssl_private.h:#if (OPENSSL_VERSION_NUMBER < 0x10100000L) && ! (defined(WIN32) || defined(WIN64))
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L && ! (defined(WIN32) || defined(WIN64))
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/ssl.c:#if !defined(OPENSSL_NO_ENGINE) || OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/ssl.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslcontext.c:#endif /* if OPENSSL_VERSION_NUMBER < 0x10100000L */
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#else /* if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER) */
> ./src/sslcontext.c:#endif /* if OPENSSL_VERSION_NUMBER < 0x10100000L */
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if (OPENSSL_VERSION_NUMBER < 0x10100000L)
> ./src/sslcontext.c:#if (OPENSSL_VERSION_NUMBER < 0x10100000L)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslcontext.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslinfo.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> ./src/sslutils.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L
> ./src/sslutils.c:#if OPENSSL_VERSION_NUMBER < 0x10100000L

Comment 4 Michael Osipov 2020-05-11 12:12:44 UTC

This is what LibreSSL 3.1.1 defines:

> [mosipov@mika-ion ~/Projekte/tomcat-native/native]$ grep -ri -E -e '#define .+_version'  /usr/local/include/openssl/
> /usr/local/include/openssl/crypto.h:#define SSLEAY_VERSION_NUMBER       OPENSSL_VERSION_NUMBER
> /usr/local/include/openssl/crypto.h:#define SSLEAY_VERSION              0
> /usr/local/include/openssl/crypto.h:#define OPENSSL_VERSION             0
> /usr/local/include/openssl/opensslv.h:#define LIBRESSL_VERSION_NUMBER 0x3010100fL
> /usr/local/include/openssl/opensslv.h:#define LIBRESSL_VERSION_TEXT   "LibreSSL 3.1.1"
> /usr/local/include/openssl/opensslv.h:#define OPENSSL_VERSION_NUMBER    0x20000000L
> /usr/local/include/openssl/opensslv.h:#define OPENSSL_VERSION_TEXT      LIBRESSL_VERSION_TEXT
> /usr/local/include/openssl/opensslv.h:#define OPENSSL_VERSION_PTEXT     " part of " OPENSSL_VERSION_TEXT
> /usr/local/include/openssl/opensslv.h:#define SHLIB_VERSION_HISTORY ""
> /usr/local/include/openssl/opensslv.h:#define SHLIB_VERSION_NUMBER "1.0.0"

Comment 5 Michael Osipov 2020-05-11 13:03:04 UTC

As soon as I revert b8649e81458194d70667952d9e26df82a79c773f I only see and the code compiles

> ./include/ssl_private.h:220:9: warning: 'OPENSSL_VERSION' macro redefined [-Wmacro-redefined]
> #define OPENSSL_VERSION                  SSLEAY_VERSION
>         ^
> /usr/local/include/openssl/crypto.h:329:9: note: previous definition is here
> #define OPENSSL_VERSION         0
>         ^
> src/ssl.c:301:9: warning: implicit declaration of function 'SSL_CTX_set_keylog_callback' is invalid in C99
>       [-Wimplicit-function-declaration]
>         SSL_CTX_set_keylog_callback(ctx, ssl_keylog_callback);
>         ^

It pretty much seems that the the change was incomplete -- as assumed.

Shall we revert for now?

Comment 6 gessel 2020-05-13 11:43:25 UTC

tomcat-native-1.2.24_1 builds successfully, thank you!

Comment 7 Michael Osipov 2020-05-13 12:34:11 UTC

I will supersede this ticket with a new, general one. Downstream patches have been applied, but upstream is still broken.