Verified with * Apache Tomcat Native library [1.2.40-dev] using APR version [1.7.4] * [OpenSSL 1.1.1t 7 Feb 2023] * Tomcat Apache Tomcat/9.0.81-dev But also verified on Windows and FreeBSD, this is a purely client code issue of ours. Consider the following: > osipovmi@deblndw024v:~/apache-tomcat-9.0.81-dev > $ ll conf/certs-localhost/key.crt > ---------- 1 osipovmi cad 3434 2023-09-29 23:05 conf/certs-localhost/key.crt Note: the permissions have been removed for demonstration purposes, it is not about the permissions, but about incomplete error reporting. Now launch: > $ openssl s_server -accept 0.0.0.0:20000 -key conf/certs-localhost/key.crt -cert conf/certs-localhost/cert.crt -pass file:conf/certs-localhost/key-password > Can't open conf/certs-localhost/key.crt for reading, Permission denied > 1:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:69:fopen('conf/certs-localhost/key.crt','r') > 1:error:2006D002:BIO routines:BIO_new_file:system lib:crypto/bio/bss_file.c:78: > unable to load server certificate private key file We can now see two OpenSSL errors: 0200100D and 2006D002: > $ openssl errstr 2006D002 > error:2006D002:BIO routines:BIO_new_file:system lib > $ openssl errstr 0200100D > error:0200100D:system library:fopen:Permission denied Now let's do this with Tomcat: > <Connector port="30001" protocol="org.apache.coyote.http11.Http11AprProtocol" > maxThreads="150" SSLEnabled="true" > maxParameterCount="1000"> > <SSLHostConfig> > <Certificate certificateKeyFile="conf/certs-localhost/key.crt" > certificateFile="conf/certs-localhost/cert.crt" > certificateChainFile="conf/cacerts.crt" > type="RSA" /> > </SSLHostConfig> > </Connector> Output: > 05-Oct-2023 09:16:13.086 INFORMATION [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-apr-30001"] > 05-Oct-2023 09:16:13.098 INFORMATION [main] org.apache.tomcat.util.net.openssl.OpenSSLUtil.getKeyManagers The certificate [conf/certs-localhost/cert.crt] or its private key [conf/certs-localhost/key.crt] could not be processed using a JSSE key manager and will be given directly to OpenSSL > 05-Oct-2023 09:16:13.099 WARNUNG [main] org.apache.tomcat.util.net.openssl.OpenSSLContext.init Error initializing SSL context > java.lang.Exception: Unable to load certificate key /net/home/osipovmi/apache-tomcat-9.0.81-dev/conf/certs-localhost/key.crt (error:0200100D:system library:fopen:Berechtigung verweigert) > at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method) > at org.apache.tomcat.util.net.openssl.OpenSSLContext.addCertificate(OpenSSLContext.java:467) > at org.apache.tomcat.util.net.openssl.OpenSSLContext.init(OpenSSLContext.java:336) > at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:268) > at org.apache.tomcat.util.net.AprEndpoint.createSSLContext(AprEndpoint.java:467) > at org.apache.tomcat.util.net.AprEndpoint.bind(AprEndpoint.java:433) > at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1326) > at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1339) > at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:654) > at org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:75) > at org.apache.catalina.connector.Connector.initInternal(Connector.java:1009) > ... According to my humble OpenSSL doc understanding (https://www.openssl.org/docs/man3.0/man3/ERR_get_error.html) OpenSSL maintains an error queue per thread which we never consume in a while loop nor do we clear the queue when not fully consumed. From a user's PoV this is unfortunate because he/she might miss important error information here. It might be worth looking at mod_ssl's ssl_log_ssl_error() to get all errors from the queue.
This has been discovered while working on Bug 66670.
Here is truly context missing: > 05-Oct-2023 11:57:22.432 WARNUNG [main] org.apache.tomcat.util.net.openssl.OpenSSLContext.init Error initializing SSL context > java.lang.Exception: Unable to load certificate key password file /net/home/osipovmi/apache-tomcat-9.0.81-dev/conf/certs-localhost/key-perm-password (error:2006D002:BIO routines:BIO_new_file:system lib) > at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method)
I would say this probably will not happen. However the Panama equivalent is a much better place to have precise (where the error actually occurs), integrated (using the same logging as the rest of Tomcat) and detailed (having access to all necessary OpenSSL error details) error reporting.
(In reply to Remy Maucherat from comment #3) > I would say this probably will not happen. How? What makes you so sure? > However the Panama equivalent is a much better place to have precise (where > the error actually occurs), integrated (using the same logging as the rest > of Tomcat) and detailed (having access to all necessary OpenSSL error > details) error reporting. This is unfeasable for a lot of users. I would expect that this would also be solved in C along with the FFM API.
It may just be a coincidence, but in this case (cannot load cert; file permission error), Tomcat detects the important error (permission error) and reports it to the client. Would it be enough to simply clear the error queue any time we consult it? Just... discard the remaining errors in the queue? Otherwise, we'd have to grab all the error codes (and strings?) and return what amounts to a compound exception to the caller. It's doable, of course, but ugly.
(In reply to Christopher Schultz from comment #5) > It may just be a coincidence, but in this case (cannot load cert; file > permission error), Tomcat detects the important error (permission error) and > reports it to the client. Well, it does not only apply to the cert loading issue, but to any OpenSSL function which might return with an error. > Would it be enough to simply clear the error queue any time we consult it? > Just... discard the remaining errors in the queue? Otherwise, we'd have to > grab all the error codes (and strings?) and return what amounts to a > compound exception to the caller. It's doable, of course, but ugly. As far as I understand the documentation you must clear the queue if you aren't going to consume it completely. Grabbing everything like in mod_ssl is, of course, the best option not to miss something.
(In reply to Michael Osipov from comment #4) > (In reply to Remy Maucherat from comment #3) > > I would say this probably will not happen. > > How? What makes you so sure? The pace of changes in tomcat-native ... > > However the Panama equivalent is a much better place to have precise (where > > the error actually occurs), integrated (using the same logging as the rest > > of Tomcat) and detailed (having access to all necessary OpenSSL error > > details) error reporting. > > This is unfeasable for a lot of users. I would expect that this would also > be solved in C along with the FFM API. Yes and no. In a few months time, you could debug/test this kind of startup issue with Java 22, before moving back to tomcat-native and a more reasonable Java version.