Bug 66297 - Undefined symbol: RAND_egd modules/ssl/.libs/libmod_ssl.a(ssl_engine_rand.o)
Summary: Undefined symbol: RAND_egd modules/ssl/.libs/libmod_ssl.a(ssl_engine_rand.o)
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Build (show other bugs)
Version: 2.5-HEAD
Hardware: Sun Solaris
: P2 regression (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-10-06 15:03 UTC by ran.mozes
Modified: 2022-10-10 09:45 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ran.mozes 2022-10-06 15:03:15 UTC
While trying to compile with the following configuration:

CFLAGS="-m32"; export CFLAGS
"./configure" \
"--prefix=/var/opt/sun/xvm/uce/opt/server" \
"--disable-proxy_express" \
"--disable-proxy_balancer" \
"--enable-cgi=static" \
"--enable-dir=static" \
"--enable-ssl=static" \
"--enable-pcre=static" \
"--enable-unixd=static" \
"--enable-alias=static" \
"--enable-setenvif=static" \
"--enable-log-config=static" \
"--enable-deflate=static" \
"--enable-filter=static" \
"--enable-socache-dbm=static" \
"--enable-mime=static" \
"--enable-authz-core=static" \
"--enable-authz-host=static" \
"--enable-proxy=static" \
"--enable-proxy-http=static" \
"--enable-rewrite=static" \
"--enable-headers=static" \
"--enable-cache=static" \
"--enable-cache-disk=static" \
"--enable-dav=static" \
"--enable-autoindex=static" \
"--enable-reqtimeout=static" \
"--enable-mime_magic=static" \
"--with-mpm=prefork" \
"--with-ssl=/var/opt/sun/xvm/uce/opt/server/" \
"--with-module=sigvalidate:/scratch/rmozes/apache_upgrade_2.4.54/mod_sigvalidate.c" \
"--with-included-apr" \
"--with-pcre=/var/opt/sun/xvm/uce/opt/server/bin/pcre-config" \
"--with-zlib=/var/opt/sun/xvm/uce/opt/server" \
"--with-expat=/usr/sfw/" \
"CFLAGS=-m32" \
"$@"

using modules: 
apr-1.7.0
apr-util-1.6.1
openssl1.1.1o (same error with openssl1.1.1q)

the make process will fail with undefined symbol RAND_egd: 

bash-3.2$ /scratch/rmozes/apache_upgrade_2.4.54/httpd-2.4.54/srclib/apr/libtool --silent --mode=link gcc -std=gnu99    -m32    -L/var/opt/sun/xvm/uce/opt/server/lib   -o httpd  modules.lo buildmark.o -export-dynamic server/libmain.la  modules/aaa/libmod_authz_host.la modules/aaa/libmod_authz_core.la modules/cache/libmod_cache.la modules/cache/libmod_cache_disk.la modules/cache/libmod_socache_dbm.la modules/core/libmod_so.la modules/filters/libmod_reqtimeout.la modules/filters/libmod_filter.la modules/filters/libmod_deflate.la modules/http/libmod_http.la modules/http/libmod_mime.la modules/loggers/libmod_log_config.la modules/metadata/libmod_mime_magic.la modules/metadata/libmod_headers.la modules/metadata/libmod_setenvif.la modules/proxy/libmod_proxy.la modules/proxy/libmod_proxy_connect.la modules/proxy/libmod_proxy_ftp.la modules/proxy/libmod_proxy_http.la modules/proxy/libmod_proxy_fcgi.la modules/proxy/libmod_proxy_scgi.la modules/proxy/libmod_proxy_uwsgi.la modules/proxy/libmod_proxy_wstunnel.la modules/proxy/libmod_proxy_ajp.la modules/ssl/libmod_ssl.la modules/arch/unix/libmod_unixd.la modules/dav/main/libmod_dav.la modules/generators/libmod_autoindex.la modules/generators/libmod_cgi.la modules/dav/fs/libmod_dav_fs.la modules/sigvalidate/libmod_sigvalidate.la modules/mappers/libmod_dir.la modules/mappers/libmod_alias.la modules/mappers/libmod_rewrite.la  server/mpm/prefork/libprefork.la  os/unix/libos.la -L/var/opt/sun/xvm/uce/opt/server/lib -R/var/opt/sun/xvm/uce/opt/server/lib -lpcre          -lz                         /scratch/rmozes/apache_upgrade_2.4.54/httpd-2.4.54/srclib/apr-util/libaprutil-1.la -lexpat /scratch/rmozes/apache_upgrade_2.4.54/httpd-2.4.54/srclib/apr/libapr-1.la -luuid -lsendfile -lrt -lsocket -lnsl -lpthread
Undefined                       first referenced
 symbol                             in file
RAND_egd                            modules/ssl/.libs/libmod_ssl.a(ssl_engine_rand.o)
ld: fatal: symbol referencing errors. No output written to .libs/httpd
collect2: ld returned 1 exit status
bash-3.2$

the symbold RAND_egd is truly not there  
bash-3.2$ nm -gDC /var/opt/sun/xvm/uce/opt/server/lib/libcrypto.so | grep -i "RAND_egd"
bash-3.2$ nm -gDC /var/opt/sun/xvm/uce/opt/server/lib/libssl.so | grep -i "RAND_egd"
bash-3.2$

I'm not sure if this is an Apache bug or openssl or rather something I should have done while compiling the openssl. 
openssl was compiled with:
./Configure solaris-x86-gcc shared threads 386 no-ssl3 --prefix=/var/opt/sun/xvm/uce/opt/server --openssldir=/var/opt/sun/xvm/uce/opt/server
CNF_CFLAGS=-pthreads -Wa,--noexecstack
CNF_EX_LIBS=-lsocket -lnsl -ldl -pthreads -lrt
Comment 1 Rainer Jung 2022-10-06 16:52:09 UTC
There seems to be an OpenSSL header file mixup.

When OpenSSL is compiled, it detects egd support. The definition of RAND_egd() in openssl/rand.h is protected by an

# ifndef OPENSSL_NO_EGD

and in (my) openssl/opensslconf.h I find:

#ifndef OPENSSL_NO_EGD
# define OPENSSL_NO_EGD
#endif

When httpd gets compiled, its configure script checks for RAND_egd via

      AC_CHECK_FUNCS([ENGINE_init ENGINE_load_builtin_engines RAND_egd])

and sets HAVE_RAND_EGD in include/ap_config_auto.h. For my build there is:

/* #undef HAVE_RAND_EGD */

Configure output and config.log also show

checking for RAND_egd... no

So it seems the OpenSSL which is found and used by configure supports RAND_egd (resp. OPENSSL_NO_EGD is undefined) but the one used for the compilation does not.

Please check configure output and the definitions in the quoted header file, to find, where things start to go wrong.
Comment 2 ran.mozes 2022-10-07 07:56:16 UTC
Thanks for looking into it Rainer. 

I double checked that and I don’t think to have conflicting versions of the openssl in the included paths. I cleaned the path /var/opt/sun/xvm/uce/opt/server/ before compilation,
then compiled the current openssl version into it. 

configure:27443: checking for OpenSSL
configure:27456: checking for user-provided OpenSSL base directory
configure:27472: result: /var/opt/sun/xvm/uce/opt/server
configure:27809: checking for OpenSSL version >= 0.9.8a
configure:27828: gcc -std=gnu99 -c -m32  -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I/var/opt/sun/xvm/uce/opt/server/include conftest.c >&5
configure:27828: $? = 0
configure:27829: result: OK

Looks the same in (my) opensslconf.h file 
#ifndef OPENSSL_NO_EGD
# define OPENSSL_NO_EGD
#endif


in (my) ap_config_auto.h it says:
/* Define to 1 if you have the `RAND_egd' function. */
#define HAVE_RAND_EGD 1

This file differs from yours. 

in config.log I see:
configure:27935: checking for RAND_egd
configure:27935: gcc -std=gnu99 -o conftest -m32  -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I/var/opt/sun/xvm/uce/opt/server/include   -L/usr/sfw//lib -L/var/opt/sun/xvm/uce/opt/server/lib conftest.c -lssl -lcrypto    -luuid -lsendfile -lrt -lsocket -lnsl  -lpthread >&5
configure:27935: $? = 0
configure:27935: result: yes
configure:27961: result: yes

Not sure if related but the file also includes this line:
ac_cv_func_RAND_egd=yes

and also found this:
include/ap_config_auto.h.in:/* Define to 1 if you have the `RAND_egd' function. */
include/ap_config_auto.h.in:#undef HAVE_RAND_EGD


So it seems the OpenSSL which is found and used by configure supports RAND_egd
(resp. OPENSSL_NO_EGD is undefined) but the one used for the compilation does
not.

Looks to me like openssl claims to support the functionality but doesn’t expose it in the '.so‘ file as it should have. 
Do you think the same? Can you find the symbols of the RAND_egd* functions in libcrypto.so or libssl.so? 

Also Apache decides to use this functionality (#define HAVE_RAND_EGD 1) although the Solaris OS does include  (see openssl doc for this: https://www.openssl.org/docs/manmaster/man3/RAND_egd.html) 
ls /dev/urandom
/dev/urandom

I wonder why in your case the compilation has decided to use /* #undef HAVE_RAND_EGD */ 

As a workaround I have manually changed ap_config_auto.h
   include/ap_config_auto.h.in:#undef HAVE_RAND_EGD
then ran the make process from start and the compilation ended successfully.
Comment 3 ran.mozes 2022-10-07 11:58:14 UTC
exchanging information also with openssl folks, they said that the configuration used in opensslconf.h and rand.h is disabling the RAND_egd by default unless openssl is compiled with "enable-egd" option (which it wasn't).

It seems likely to me that a bug in the Apache build process is incorrectly detecting that EGD is enabled when it is not. 

The other option with 2 sets of OpenSSL headers seems unlikely to me because on my build env the Apache config process should have taken whatever is there from one location "--with-ssl=/var/opt/sun/xvm/uce/opt/server/"
This location was cleaned before the process and then openssl was built with the same prefix as mentioned before. 

bash-3.2$ sudo find /var/opt/ -name opensslconf.h
Password:
/var/opt/sun/xvm/uce/opt/server/include/openssl/opensslconf.h

only one instance found
Comment 4 Rainer Jung 2022-10-10 09:45:06 UTC
> configure:27443: checking for OpenSSL
> configure:27456: checking for user-provided OpenSSL base directory
> configure:27472: result: /var/opt/sun/xvm/uce/opt/server
> configure:27809: checking for OpenSSL version >= 0.9.8a
> configure:27828: gcc -std=gnu99 -c -m32  -DSOLARIS2=10
> -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE
> -I/var/opt/sun/xvm/uce/opt/server/include conftest.c >&5
> configure:27828: $? = 0
> configure:27829: result: OK

Good.

> Looks the same in (my) opensslconf.h file
> #ifndef OPENSSL_NO_EGD
> # define OPENSSL_NO_EGD
> #endif

Good.

> in (my) ap_config_auto.h it says:
> /* Define to 1 if you have the `RAND_egd' function. */
> #define HAVE_RAND_EGD 1

Bad, but it comes from ...

> in config.log I see:
> configure:27935: checking for RAND_egd
> configure:27935: gcc -std=gnu99 -o conftest -m32  -DSOLARIS2=10
> -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE
> -I/var/opt/sun/xvm/uce/opt/server/include   -L/usr/sfw//lib
> -L/var/opt/sun/xvm/uce/opt/server/lib conftest.c -lssl -lcrypto    -luuid
> -lsendfile -lrt -lsocket -lnsl  -lpthread >&5
> configure:27935: $? = 0
> configure:27935: result: yes
> configure:27961: result: yes

So this is the culprit and what we ned to understand.

> Not sure if related but the file also includes this line:
> ac_cv_func_RAND_egd=yes

That is also a consequence of configure detecting RAND_egd.

> and also found this:
> include/ap_config_auto.h.in:/* Define to 1 if you have the `RAND_egd' function.
> */
> include/ap_config_auto.h.in:#undef HAVE_RAND_EGD

That is also a consequence of configure detecting RAND_egd.

> So it seems the OpenSSL which is found and used by configure supports RAND_egd
> (resp. OPENSSL_NO_EGD is undefined) but the one used for the compilation does
> not.

Yes.

> Looks to me like openssl claims to support the functionality but doesn’t expose
> it in the '.so‘ file as it should have.
> Do you think the same? Can you find the symbols of the RAND_egd* functions in
> libcrypto.so or libssl.so?

No it doesn't have the symbol either.

BUT: For instance on my Solaris 11 Sparc system, there is a local OpenSSL installation under /lib/64/libcrypto.so.1.0.0 which *does* have the RAND_egd symbol. The local file /usr/include/openssl/opensslconf.h does not define OPENSSL_NO_EGD and /usr/include/openssl/rand.h declares RAND_egd. So in my case the system installation of openssl does support RAND_egd, but my custom OpenSSL, which I use to build httpd, does not.

To make sure, that httpd finds all info for your custom openssl - header files and libs - before the ones from the operating system, the include and linker paths need to be set up correctly during configure and build.

Since during the configure check on your system the gcc flags are:

gcc -std=gnu99 -o conftest -m32  -DSOLARIS2=10
-D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE
-I/var/opt/sun/xvm/uce/opt/server/include   -L/usr/sfw//lib
-L/var/opt/sun/xvm/uce/opt/server/lib conftest.c -lssl -lcrypto    -luuid
-lsendfile -lrt -lsocket -lnsl  -lpthread

we can see, that it would eg. use libs from /usr/sfw/lib before your own openssl libs.

Therefore I think it makes sense to do the litle investigation that I suggest further below.

In addition: what are your configure flags plus additional env vars you set? 

> Also Apache decides to use this functionality (#define HAVE_RAND_EGD 1)
> although the Solaris OS does include  (see openssl doc for this:
> https://www.openssl.org/docs/manmaster/man3/RAND_egd.html)
> ls /dev/urandom
> /dev/urandom

Using egd - if detected as existing - seems to have higher priority than just /dev/urandom.

> I wonder why in your case the compilation has decided to use /* #undef
> HAVE_RAND_EGD */
> 
> As a workaround I have manually changed ap_config_auto.h
>     include/ap_config_auto.h.in:#undef HAVE_RAND_EGD
> then ran the make process from start and the compilation ended successfully.

OK, the compilation result will be fine, the workaround is OK, but of course it would be better to understand why it goes wrong without woraround.

So if you are still interested in fixing it really, we can have a look at what the above

 > configure:27935:

gcc -std=gnu99 -o conftest -m32  -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I/var/opt/sun/xvm/uce/opt/server/include   -L/usr/sfw//lib -L/var/opt/sun/xvm/uce/opt/server/lib conftest.c -lssl -lcrypto -luuid -lsendfile -lrt -lsocket -lnsl  -lpthread >&5

compiles and why it succeeeds.

There are several ways of doing it, but the following should be easy:

- edit your configure script, replace "RAND_egd" by "RAND_egd_unknown"

There should be only one line in configure that contains the function name:

       for ac_func in ENGINE_init ENGINE_load_builtin_engines RAND_egd

After you replace it with the name of the function that definitely does not exist, rerun configure. Since this time the detection fails, configure will also output the full text of the C program it tried to compile. That is helpful, because we can pick that C source code, put it into a file conftest.c and run the compilation ourselfs with the above

gcc -std=gnu99 -o conftest -m32  -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I/var/opt/sun/xvm/uce/opt/server/include   -L/usr/sfw//lib -L/var/opt/sun/xvm/uce/opt/server/lib conftest.c -lssl -lcrypto -luuid -lsendfile -lrt -lsocket -lnsl  -lpthread

Of course we would replace RAND_egd_unknown by the original RAND_egd in conftest.c.

For example in my case, configure outputs:

...
checking for RAND_egd
gcc ...
Undefined                       first referenced
  symbol                             in file
RAND_egd                            /var/tmp//ccIg85bK.o
ld: fatal: Symbol referencing errors. No output written to conftest
collect2: error: ld returned 1 exit status
$? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define __EXTENSIONS__ 1
| #define _ALL_SOURCE 1
| #define _GNU_SOURCE 1
| #define _POSIX_PTHREAD_SEMANTICS 1
| #define _TANDEM_SOURCE 1
| #define STDC_HEADERS 1
| #define HAVE_STRING_H 1
| #define HAVE_LIMITS_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_SYS_SOCKET_H 1
| #define HAVE_PWD_H 1
| #define HAVE_GRP_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_SYS_PROCESSOR_H 1
| #define HAVE_SYS_SEM_H 1
| #define HAVE_SYS_SDT_H 1
| #define HAVE_SYS_LOADAVG_H 1
| #define HAVE_SYS_WAIT_H 1
| #define HAVE_GETPWNAM 1
| #define HAVE_GETGRNAM 1
| #define HAVE_INITGROUPS 1
| #define HAVE_GETPGID 1
| #define HAVE_FOPEN64 1
| #define HAVE_GETLOADAVG 1
| #define HAVE_SYS_TIMES_H 1
| #define HAVE_TIMES 1
| #define HAVE_OPENSSL_ENGINE_H 1
| #define HAVE_SSL_CTX_NEW 1
| #define HAVE_OPENSSL_INIT_SSL 1
| #define HAVE_ENGINE_INIT 1
| #define HAVE_ENGINE_LOAD_BUILTIN_ENGINES 1
| /* end confdefs.h.  */
| /* Define RAND_egd to an innocuous variant, in case <limits.h> declares RAND_egd.
|    For example, HP-UX 11i <limits.h> declares gettimeofday.  */
| #define RAND_egd innocuous_RAND_egd
|
| /* System header to define __stub macros and hopefully few prototypes,
|     which can conflict with char RAND_egd (); below.
|     Prefer <limits.h> to <assert.h> if __STDC__ is defined, since
|     <limits.h> exists even on freestanding compilers.  */
|
| #ifdef __STDC__
| # include <limits.h>
| #else
| # include <assert.h>
| #endif
|
| #undef RAND_egd
|
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char RAND_egd ();
| /* The GNU C library defines this for functions which it implements
|     to always fail with ENOSYS.  Some functions are actually named
|     something starting with __ and the normal name is an alias.  */
| #if defined __stub_RAND_egd || defined __stub___RAND_egd
| choke me
| #endif
|
| int
| main ()
| {
| return RAND_egd ();
|   ;
|   return 0;
| }
...

The text block with the "| " in front of the lines is the conftest.c contents. But it is better you create it on your system and not just take the one from mine.

Once you have your conftest.c and run gcc - which in your case should succeed although we do not know yet why, you can do some checks:

Check with "nm" and "ldd" whether the symbol is referenced by the created a.out and where it might come from? If the symbol gets used, you should be able to find the library that provided it. If the symbol is not used, it might be defined as a macro somewhere (although the above test code tries to rule that out). Nevertheless you can then use "gcc -E" to only resolve all header files and definitions in conftest.c and look in the result for RAND_egd.

I hope that's understandable!

Best regards,

Rainer