Self-signed certificates cause an unpatched Apache 2.4.9 to fail to start when SSLCACertificateFile is left unspecified. This worked correctly under Apache 2.4.7. -------------------------------------------- The following error is emitted to error_log: -------------------------------------------- [Mon Apr 14 15:06:11.486441 2014] [suexec:notice] [pid 27131] AH01232: suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec) [Mon Apr 14 15:06:11.491588 2014] [ssl:emerg] [pid 27132] AH02562: Failed to configure certificate cent5ssl.loc:443:0 (with chain), check /tmp/ssl-keys/server.crt [Mon Apr 14 15:06:11.491635 2014] [ssl:emerg] [pid 27132] SSL Library Error: error:0906D06C:PEM routines:PEM_read_bio:no start line (Expecting: DH PARAMETERS) -- Bad file contents or format - or even just a forgotten SSLCertificateKeyFile? [Mon Apr 14 15:06:11.491647 2014] [:emerg] [pid 27132] AH00020: Configuration Failed, exiting -------------------------------------------- To duplicate, I did the following: -------------------------------------------- 1. Downloaded Apache 2.4.9 2. Downloaded APR 1.5.0 3. Downloaded APR-util 1.5.3 4. Extracted Apache tarball 5. Extracted both APR libraries into srclib directory 6. Ran the following configure line: ./configure --disable-v4-mapped --enable-access-compat=static --enable-actions=static --enable-alias=static --enable-auth_basic=static --enable-authn_core=static --enable-authn_file=static --enable-authz_core=static --enable-authz_groupfile=static --enable-authz_host=static --enable-authz_user=static --enable-autoindex=static --enable-cgi=static --enable-deflate=static --enable-dir=static --enable-expires=static --enable-filter=static --enable-headers=static --enable-include=static --enable-info=static --enable-log_config=static --enable-logio=static --enable-mime=static --enable-modules=none --enable-negotiation=static --enable-proxy=static --enable-proxy-connect=static --enable-proxy-http=static --enable-rewrite=static --enable-setenvif=static --enable-slotmem_shm=static --enable-socache_dbm=static --enable-socache_shmcb=static --enable-ssl=static --enable-status=static --enable-suexec=static --enable-unixd=static --enable-userdir=static --prefix=/usr/local/apache --with-included-apr --with-mpm=prefork --with-pcre=/opt/pcre --with-ssl=/usr --with-suexec-caller=nobody --with-suexec-docroot=/ --with-suexec-gidmin=100 --with-suexec-logfile=/usr/local/apache/logs/suexec_log --with-suexec-uidmin=100 --with-suexec-userdir=public_html 7. Generated self-signed certificate and key: mkdir /tmp/ssl-keys cd /tmp/ssl-keys openssl req -new -x509 -nodes -out server.crt -keyout server.key -extensions usr_cert 8. Created a virtual host in Apache, then added the following SSL options: SSLEngine on SSLCertificateFile /tmp/ssl-keys/server.crt SSLCertificateKeyFile /tmp/ssl-keys/server.key 9. Validated that this _works_ with Apache 2.4.7 using OpenSSL 0.9.8e-fips-rhel5 (CentOS 5.10) 10. Validated that this _breaks_ with Apache 2.4.9 using OpenSSL 0.9.8e-fips-rhel5 (CentOS 5.10) 11. Validated that this _works_ with Apache 2.4.9 using OpenSSL 1.0.1e-fips (CentOS 6.5)
Here is a working version of Apache debugging output using 2.4.9 using OpenSSL 1.0.1e: [Mon Apr 14 19:27:36.436555 2014] [ssl:info] [pid 7676] AH01887: Init: Initializing (virtual) servers for SSL [Mon Apr 14 19:27:36.436896 2014] [ssl:info] [pid 7676] AH01914: Configuring server cent6ssl.loc:443 for SSL protocol [Mon Apr 14 19:27:36.437045 2014] [ssl:debug] [pid 7676] ssl_engine_init.c(312): AH01893: Configuring TLS extension handling [Mon Apr 14 19:27:36.437614 2014] [ssl:debug] [pid 7676] ssl_util_ssl.c(343): AH02412: [cent6ssl.loc:443] Cert matches for name 'cent6ssl.loc' [subject: emailAddress=kurt.newman@cpanel.net,CN=cent6ssl.loc,OU=N/A,O=Cent 6 SSL rev2\\, Inc,L=Houston,ST=Texas,C=US / issuer: emailAddress=kurt.newman@cpanel.net,CN=cent6ssl.loc,OU=N/A,O=Cent 6 SSL rev2\\, Inc,L=Houston,ST=Texas,C=US / serial: E1CFBAAB372443FB / notbefore: Apr 14 20:01:19 2014 GMT / notafter: May 14 20:01:19 2014 GMT] [Mon Apr 14 19:27:36.437641 2014] [ssl:info] [pid 7676] AH02568: Certificate and private key cent6ssl.loc:443:0 configured from /tmp/ssl-keys/server.crt and /tmp/ssl-keys/server.key [Mon Apr 14 19:27:36.437841 2014] [ssl:info] [pid 7676] AH01876: mod_ssl/2.4.9 compiled against Server: Apache/2.4.9, Library: OpenSSL/1.0.1e [Mon Apr 14 19:27:36.437858 2014] [suexec:notice] [pid 7676] AH01232: suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec) [Mon Apr 14 19:27:36.444022 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(396): AH00821: shmcb_init allocated 1024000 bytes of shared memory [Mon Apr 14 19:27:36.444044 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(412): AH00822: for 1023944 bytes (1024000 including header), recommending 32 subcaches, 177 indexes each [Mon Apr 14 19:27:36.444051 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(445): AH00824: shmcb_init_memory choices follow [Mon Apr 14 19:27:36.444056 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(447): AH00825: subcache_num = 32 [Mon Apr 14 19:27:36.444061 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(449): AH00826: subcache_size = 31992 [Mon Apr 14 19:27:36.444066 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(451): AH00827: subcache_data_offset = 4264 [Mon Apr 14 19:27:36.444071 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(453): AH00828: subcache_data_size = 27728 [Mon Apr 14 19:27:36.444076 2014] [socache_shmcb:debug] [pid 7677] mod_socache_shmcb.c(455): AH00829: index_num = 177 [Mon Apr 14 19:27:36.444156 2014] [socache_shmcb:info] [pid 7677] AH00830: Shared memory socache initialised [Mon Apr 14 19:27:36.444162 2014] [ssl:info] [pid 7677] AH01887: Init: Initializing (virtual) servers for SSL [Mon Apr 14 19:27:36.444344 2014] [ssl:info] [pid 7677] AH01914: Configuring server cent6ssl.loc:443 for SSL protocol [Mon Apr 14 19:27:36.444480 2014] [ssl:debug] [pid 7677] ssl_engine_init.c(312): AH01893: Configuring TLS extension handling [Mon Apr 14 19:27:36.444874 2014] [ssl:debug] [pid 7677] ssl_util_ssl.c(343): AH02412: [cent6ssl.loc:443] Cert matches for name 'cent6ssl.loc' [subject: emailAddress=kurt.newman@cpanel.net,CN=cent6ssl.loc,OU=N/A,O=Cent 6 SSL rev2\\, Inc,L=Houston,ST=Texas,C=US / issuer: emailAddress=kurt.newman@cpanel.net,CN=cent6ssl.loc,OU=N/A,O=Cent 6 SSL rev2\\, Inc,L=Houston,ST=Texas,C=US / serial: E1CFBAAB372443FB / notbefore: Apr 14 20:01:19 2014 GMT / notafter: May 14 20:01:19 2014 GMT] [Mon Apr 14 19:27:36.444895 2014] [ssl:info] [pid 7677] AH02568: Certificate and private key cent6ssl.loc:443:0 configured from /tmp/ssl-keys/server.crt and /tmp/ssl-keys/server.key [Mon Apr 14 19:27:36.445092 2014] [ssl:info] [pid 7677] AH01876: mod_ssl/2.4.9 compiled against Server: Apache/2.4.9, Library: OpenSSL/1.0.1e [Mon Apr 14 19:27:36.446296 2014] [mpm_prefork:notice] [pid 7677] AH00163: Apache/2.4.9 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 configured -- resuming normal operations [Mon Apr 14 19:27:36.446312 2014] [mpm_prefork:info] [pid 7677] AH00164: Server built: Apr 14 2014 19:27:03 [Mon Apr 14 19:27:36.446337 2014] [core:notice] [pid 7677] AH00094: Command line: '/usr/local/apache/bin/httpd -D SSL' [Mon Apr 14 19:27:36.446349 2014] [mpm_prefork:debug] [pid 7677] prefork.c(995): AH00165: Accept mutex: sysvsem (default: sysvsem) [Mon Apr 14 19:27:36.446348 2014] [proxy:debug] [pid 7680] proxy_util.c(1766): AH00925: initializing worker proxy:reverse shared [Mon Apr 14 19:27:36.446381 2014] [proxy:debug] [pid 7680] proxy_util.c(1808): AH00927: initializing worker proxy:reverse local [Mon Apr 14 19:27:36.446419 2014] [proxy:debug] [pid 7680] proxy_util.c(1859): AH00931: initialized single connection worker in child 7680 for (*) [Mon Apr 14 19:27:36.446799 2014] [proxy:debug] [pid 7683] proxy_util.c(1766): AH00925: initializing worker proxy:reverse shared [Mon Apr 14 19:27:36.446829 2014] [proxy:debug] [pid 7683] proxy_util.c(1808): AH00927: initializing worker proxy:reverse local [Mon Apr 14 19:27:36.446862 2014] [proxy:debug] [pid 7683] proxy_util.c(1859): AH00931: initialized single connection worker in child 7683 for (*) [Mon Apr 14 19:27:36.447392 2014] [proxy:debug] [pid 7681] proxy_util.c(1766): AH00925: initializing worker proxy:reverse shared [Mon Apr 14 19:27:36.447423 2014] [proxy:debug] [pid 7681] proxy_util.c(1808): AH00927: initializing worker proxy:reverse local [Mon Apr 14 19:27:36.447455 2014] [proxy:debug] [pid 7681] proxy_util.c(1859): AH00931: initialized single connection worker in child 7681 for (*) [Mon Apr 14 19:27:36.447943 2014] [proxy:debug] [pid 7679] proxy_util.c(1766): AH00925: initializing worker proxy:reverse shared [Mon Apr 14 19:27:36.447994 2014] [proxy:debug] [pid 7679] proxy_util.c(1808): AH00927: initializing worker proxy:reverse local [Mon Apr 14 19:27:36.448029 2014] [proxy:debug] [pid 7679] proxy_util.c(1859): AH00931: initialized single connection worker in child 7679 for (*) [Mon Apr 14 19:27:36.448521 2014] [proxy:debug] [pid 7682] proxy_util.c(1766): AH00925: initializing worker proxy:reverse shared [Mon Apr 14 19:27:36.448550 2014] [proxy:debug] [pid 7682] proxy_util.c(1808): AH00927: initializing worker proxy:reverse local [Mon Apr 14 19:27:36.448584 2014] [proxy:debug] [pid 7682] proxy_util.c(1859): AH00931: initialized single connection worker in child 7682 for (*)
Here is a working version of Apache debugging output using 2.4.9 using OpenSSL 0.9.8e: [Mon Apr 14 19:25:48.977675 2014] [ssl:info] [pid 14771] AH01887: Init: Initializing (virtual) servers for SSL [Mon Apr 14 19:25:48.977884 2014] [ssl:info] [pid 14771] AH01914: Configuring server cent5ssl.loc:443 for SSL protocol [Mon Apr 14 19:25:48.978360 2014] [ssl:debug] [pid 14771] ssl_util_ssl.c(343): AH02412: [cent5ssl.loc:443] Cert matches for name 'cent5ssl.loc' [subject: emailAddress=kurt.newman@cpanel.net,CN=cent5ssl.loc,OU=N/A,O=Cent5 SSL rev1\\, Inc.,L=Houston,ST=Texas,C=US / issuer: emailAddress=kurt.newman@cpanel.net,CN=cent5ssl.loc,OU=N/A,O=Cent5 SSL rev1\\, Inc.,L=Houston,ST=Texas,C=US / serial: 9BF182A0DE63A80D / notbefore: Apr 14 20:06:01 2014 GMT / notafter: May 14 20:06:01 2014 GMT] [Mon Apr 14 19:25:48.978383 2014] [ssl:info] [pid 14771] AH02568: Certificate and private key cent5ssl.loc:443:0 configured from /tmp/ssl-keys/server.crt and /tmp/ssl-keys/server.key [Mon Apr 14 19:25:48.978462 2014] [ssl:info] [pid 14771] AH01876: mod_ssl/2.4.9 compiled against Server: Apache/2.4.9, Library: OpenSSL/0.9.8e-rhel5 [Mon Apr 14 19:25:48.978477 2014] [suexec:notice] [pid 14771] AH01232: suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec) [Mon Apr 14 19:25:48.982689 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(396): AH00821: shmcb_init allocated 1024000 bytes of shared memory [Mon Apr 14 19:25:48.982703 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(412): AH00822: for 1023944 bytes (1024000 including header), recommending 32 subcaches, 177 indexes each [Mon Apr 14 19:25:48.982709 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(445): AH00824: shmcb_init_memory choices follow [Mon Apr 14 19:25:48.982714 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(447): AH00825: subcache_num = 32 [Mon Apr 14 19:25:48.982718 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(449): AH00826: subcache_size = 31992 [Mon Apr 14 19:25:48.982723 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(451): AH00827: subcache_data_offset = 4264 [Mon Apr 14 19:25:48.982727 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(453): AH00828: subcache_data_size = 27728 [Mon Apr 14 19:25:48.982732 2014] [socache_shmcb:debug] [pid 14772] mod_socache_shmcb.c(455): AH00829: index_num = 177 [Mon Apr 14 19:25:48.982824 2014] [socache_shmcb:info] [pid 14772] AH00830: Shared memory socache initialised [Mon Apr 14 19:25:48.982831 2014] [ssl:info] [pid 14772] AH01887: Init: Initializing (virtual) servers for SSL [Mon Apr 14 19:25:48.982896 2014] [ssl:info] [pid 14772] AH01914: Configuring server cent5ssl.loc:443 for SSL protocol [Mon Apr 14 19:25:48.983095 2014] [ssl:emerg] [pid 14772] AH02562: Failed to configure certificate cent5ssl.loc:443:0 (with chain), check /tmp/ssl-keys/server.crt [Mon Apr 14 19:25:48.983132 2014] [ssl:emerg] [pid 14772] SSL Library Error: error:0906D06C:PEM routines:PEM_read_bio:no start line (Expecting: DH PARAMETERS) -- Bad file contents or format - or even just a forgotten SSLCertificateKeyFile? [Mon Apr 14 19:25:48.983143 2014] [:emerg] [pid 14772] AH00020: Configuration Failed, exiting
Sorry, Comment #2 is a BROKEN version, not working.
Created attachment 31530 [details] Also clear the error queue before calling SSL_CTX_use_certificate[_chain]_file Thanks a lot for your detailed report, which was very helpful when tracking down the problem. It's the following issue in OpenSSL, in the end: https://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=1513 which was fixed in OpenSSL 0.9.8h: *) Clear error queue in SSL_CTX_use_certificate_chain_file() Clear the error queue to ensure that error entries left from older function calls do not interfere with the correct operation. [Lutz Jaenicke, Erik de Castro Lopo] We can add a workaround for this in mod_ssl, see the attached patch. Could you give it a try and report back if it solves the issue for you?
The patch fixes the issue for me.
I will get back to your later this evening about this.
This resolved the issue.
Question. Is it safe to apply this patch to any version of OpenSSL, or should it really only be applied to versions less than 0.9.8h?
(In reply to Kurt Newman from comment #8) > Question. Is it safe to apply this patch to any version of OpenSSL, or > should it really only be applied to versions less than 0.9.8h? Yes, it is safe to apply to mod_ssl (in 2.4.8 or later) irrespective of the OpenSSL version you're compiling against. I have committed this fix to trunk with r1588427 and added a backport proposal for 2.4.x with r1588430 in the meantime.
Thank you very much for your help. What's the typical policy for making something as resolved? The dev, bug creator, or someone else entirely?
It has now been committed to the 2.4.x branch with r1588496 and will therefore be in 2.4.10. (In reply to Kurt Newman from comment #10) > What's the typical policy for making something as resolved? The status will be set to RESOLVED when 2.4.10 is released (until then, we leave it open, but note the FixedInTrunk keyword).
*** Bug 56566 has been marked as a duplicate of this bug. ***
Fixed and released in 2.4.10
I'm having exactly the same problem while trying to insert Let's Encrypt certificate and chain into one file using SSLCertificateFile. Debian Jessie, Apache 2.4.10, OpenSSL 1.0.2k.
Sending more info. It only happens when CA certificate is before domain certificate, this is the error message: AH00016: Configuration Failed It works when i place CA certificate after the domain certificate BUT in this case, www.ssllabs.com reports incomplete chain.
Looks like Apache thinks that key file is for CA certificate.
Just ignore me, my fault, sorry.