Bug 47808

Summary: Child process core dumps when enabling CRL
Product: Apache httpd-2 Reporter: Lars Ove Claesson <lars.ove.claesson>
Component: mod_sslAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED LATER    
Severity: major Keywords: MassUpdate
Priority: P2    
Version: 2.2.13   
Target Milestone: ---   
Hardware: Sun   
OS: Solaris   
Attachments: Stack trace of the core file
The output of the requested commands.
The contents of the CRL.

Description Lars Ove Claesson 2009-09-09 03:07:08 UTC
Set up is an Apache 2.2.13 running SSL with client certificate authentication. When running without a defined CRL, it works fine. The client is asked for his certificate in the browser and validation is working.

When configuring a CRL for the domain, however, the client is asked for his certificate, and after a few seconds the connection is closed. According to the log file, the child process handling the connection core dumps. The contents of the CRL-file (generated by an external system) is validated with an openssl dump, and it looks fine. The file has been reencoded from DER to PEM using openssl, as the external system generates DER, only.
Comment 1 Ruediger Pluem 2009-09-09 04:01:06 UTC
Please provide a stacktrace of your core dump. See http://httpd.apache.org/dev/debugging.html.
Comment 2 Lars Ove Claesson 2009-09-09 06:58:33 UTC
Created attachment 24238 [details]
Stack trace of the core file

Please note that I have been permitted to supply the certificates and the rest of the configuration files from my security departement, as this is a test environment. Please let me know if this is necessary.
Comment 3 Lars Ove Claesson 2009-09-09 07:01:39 UTC
This issue was also present in version 2.2.11.
Comment 4 William A. Rowe Jr. 2009-09-09 07:05:45 UTC
Pasting the relevant bit of Lars' backtrace

#0  0xfe74c124 in ASN1_UTCTIME_print () from /usr/local/ssl/lib/libcrypto.so.0.9.8
(gdb) where
#0  0xfe74c124 in ASN1_UTCTIME_print () from /usr/local/ssl/lib/libcrypto.so.0.9.8
#1  0xfe861c60 in ssl_callback_SSLVerify_CRL (ok=1, ctx=0xffbff018, c=0x1a8ae0) at ssl_engine_kernel.c:1498
#2  0xfe861fbc in ssl_callback_SSLVerify (ok=1, ctx=0xffbff018) at ssl_engine_kernel.c:1361
#3  0xfe76386c in internal_verify () from /usr/local/ssl/lib/libcrypto.so.0.9.8
#4  0xfe7641e4 in X509_verify_cert () from /usr/local/ssl/lib/libcrypto.so.0.9.8
Comment 5 Ruediger Pluem 2009-09-09 08:01:44 UTC
When you have opened the core dump with gdb please issue the following commands:

frame 1
print bio
print crl
print crl->crl
print crl->crl->lastUpdate
print crl->crl->nextUpdate
print *(crl->crl->lastUpdate)
print *(crl->crl->nextUpdate)


Furthermore the crash will go away once you leave the debug log level.
Comment 6 Lars Ove Claesson 2009-09-09 08:22:14 UTC
Created attachment 24239 [details]
The output of the requested commands.
Comment 7 Lars Ove Claesson 2009-09-09 08:23:18 UTC
Not quite sure if I understood you correctly. If I disable debug log level, which is defined inside the ssl virtual host section, it defaults back to the warn level, defined globally. But the core dump still occurs.
Comment 8 Ruediger Pluem 2009-09-09 12:46:14 UTC
(In reply to comment #7)
> Not quite sure if I understood you correctly. If I disable debug log level,
> which is defined inside the ssl virtual host section, it defaults back to the
> warn level, defined globally. But the core dump still occurs.

Hm. I guess the crash happens now somewhat later in the code. Can you please crosscheck the stacktrace of the dumps that get created with debug turned off?
Nevertheless the following patch should fix the segfault you reported and give better output.

Index: modules/ssl/ssl_engine_kernel.c
===================================================================
--- modules/ssl/ssl_engine_kernel.c     (Revision 813083)
+++ modules/ssl/ssl_engine_kernel.c     (Arbeitskopie)
@@ -1457,10 +1457,20 @@
             X509_NAME_print(bio, issuer, 0);

             BIO_printf(bio, ", lastUpdate: ");
-            ASN1_UTCTIME_print(bio, X509_CRL_get_lastUpdate(crl));
+            if (X509_CRL_get_lastUpdate(crl)) {
+                ASN1_UTCTIME_print(bio, X509_CRL_get_lastUpdate(crl));
+            }
+            else {
+                BIO_printf(bio, "NULL");
+            }

             BIO_printf(bio, ", nextUpdate: ");
-            ASN1_UTCTIME_print(bio, X509_CRL_get_nextUpdate(crl));
+            if (X509_CRL_get_nextUpdate(crl)) {
+                ASN1_UTCTIME_print(bio, X509_CRL_get_nextUpdate(crl));
+            }
+            else {
+                BIO_printf(bio, "NULL");
+            }

             n = BIO_read(bio, buff, sizeof(buff) - 1);
             buff[n] = '\0';
@@ -1492,9 +1502,9 @@
         /*
          * Check date of CRL to make sure it's not expired
          */
-        i = X509_cmp_current_time(X509_CRL_get_nextUpdate(crl));
-
-        if (i == 0) {
+        if ((X509_CRL_get_nextUpdate(crl) == NULL)
+            || ((i = X509_cmp_current_time(X509_CRL_get_nextUpdate(crl)))
+                == 0)) {
             ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s,
                          "Found CRL has invalid nextUpdate field");



But your debug outputs show that the nextUpdate field of your CRL is empty which is IMHO bad. So I guess your CRL needs fixing as well.
Comment 9 Eric Covener 2009-09-09 13:33:14 UTC
> But your debug outputs show that the nextUpdate field of your CRL is empty
> which is IMHO bad. So I guess your CRL needs fixing as well.

This is permitted by RFC3280 and openssl can generate the CRL without this field.

   TBSCertList  ::=  SEQUENCE  {
        version                 Version OPTIONAL,
                                     -- if present, MUST be v2
        signature               AlgorithmIdentifier,
        issuer                  Name,
        thisUpdate              Time,
        nextUpdate              Time OPTIONAL,
        revokedCertificates     SEQUENCE OF SEQUENCE  {
Comment 10 Ruediger Pluem 2009-09-09 14:05:57 UTC
(In reply to comment #9)
> > But your debug outputs show that the nextUpdate field of your CRL is empty
> > which is IMHO bad. So I guess your CRL needs fixing as well.
> 
> This is permitted by RFC3280 and openssl can generate the CRL without this
> field.
> 
>    TBSCertList  ::=  SEQUENCE  {
>         version                 Version OPTIONAL,
>                                      -- if present, MUST be v2
>         signature               AlgorithmIdentifier,
>         issuer                  Name,
>         thisUpdate              Time,
>         nextUpdate              Time OPTIONAL,
>         revokedCertificates     SEQUENCE OF SEQUENCE  {

Thanks for the info, but how should mod_ssl behave in this case? My patch would cause it to error out. Should we treat the CRL as expired or valid or what?
Comment 11 Eric Covener 2009-09-09 15:18:38 UTC
(In reply to comment #10)
> (In reply to comment #9)
> > > But your debug outputs show that the nextUpdate field of your CRL is empty
> > > which is IMHO bad. So I guess your CRL needs fixing as well.
> > 
> > This is permitted by RFC3280 and openssl can generate the CRL without this
> > field.
> > 
> >    TBSCertList  ::=  SEQUENCE  {
> >         version                 Version OPTIONAL,
> >                                      -- if present, MUST be v2
> >         signature               AlgorithmIdentifier,
> >         issuer                  Name,
> >         thisUpdate              Time,
> >         nextUpdate              Time OPTIONAL,
> >         revokedCertificates     SEQUENCE OF SEQUENCE  {
> 
> Thanks for the info, but how should mod_ssl behave in this case? My patch would
> cause it to error out. Should we treat the CRL as expired or valid or what?


Whoops, it's more complicated, section 5.0:

Conforming CAs are not required to issue CRLs if other revocation or
   certificate status mechanisms are provided.  When CRLs are issued,
   the CRLs MUST be version 2 CRLs, include the date by which the next
   CRL will be issued in the nextUpdate field (section 5.1.2.5), include
   the CRL number extension (section 5.2.3), and include the authority
   key identifier extension (section 5.2.1).

later:

This profile requires inclusion of nextUpdate in all CRLs issued by
   conforming CRL issuers
...
 The behavior of clients processing
   CRLs which omit nextUpdate is not specified by this profile.


Iff there's no "version" extension in the CRL I suspect we should treat nextUpdate == NULL as valid, but version >1 and nextUpdate == NULL looks like it should be configurable
Comment 12 Lars Ove Claesson 2009-09-09 15:23:20 UTC
> Thanks for the info, but how should mod_ssl behave in this case? My patch would
> cause it to error out. Should we treat the CRL as expired or valid or what?

Please note that this CRL is read by another system today - a Cisco CSS. I'm
working on moving this part of the functionality to Apache. I will not
interfere with your discussion on the RFC as it is not my field, but I'm pretty
sure that the CRL is being treated as valid - an "open" expiry date, so to
speak.

Regarding the stack trace of the cores with the debug parameter off, I will
perform them tomorrow, as I suddenly have no access to the system core dump
directory. Can't seem to override where to dump the core in the Apache config
file (parameter has no effect).
Comment 13 Lars Ove Claesson 2009-09-09 15:31:04 UTC
Created attachment 24242 [details]
The contents of the CRL.

This file shows the contents of the CRL. Don't know if it shows anything you haven't already figured out.
Comment 14 Lars Ove Claesson 2009-09-11 01:05:02 UTC
I finally got to run the stack trace on a warn log level.

Seems the same (with a slight diff in line number on the last one):

#0  0xfe762fc4 in X509_cmp_time () from /usr/local/ssl/lib/libcrypto.so.0.9.8
#1  0xfe861b80 in ssl_callback_SSLVerify_CRL (ok=1, ctx=0xffbfef48, c=0x19ae50) at ssl_engine_kernel.c:1530
#2  0xfe861fbc in ssl_callback_SSLVerify (ok=1, ctx=0xffbfef48) at ssl_engine_kernel.c:1361
#3  0xfe76386c in internal_verify () from /usr/local/ssl/lib/libcrypto.so.0.9.8
#4  0xfe7641e4 in X509_verify_cert () from /usr/local/ssl/lib/libcrypto.so.0.9.8
Comment 15 Ruediger Pluem 2009-09-11 13:39:21 UTC
(In reply to comment #14)
> I finally got to run the stack trace on a warn log level.
> 
> Seems the same (with a slight diff in line number on the last one):
> 
> #0  0xfe762fc4 in X509_cmp_time () from /usr/local/ssl/lib/libcrypto.so.0.9.8
> #1  0xfe861b80 in ssl_callback_SSLVerify_CRL (ok=1, ctx=0xffbfef48, c=0x19ae50)
> at ssl_engine_kernel.c:1530

That is what I expected. Did you test the patch?
Admittedly it only removes the segfault but does not work with your CRL. See the comments from Eric regarding the discussion what should be the right thing to do in this case (empty next update field).
This still needs to be decided and worked out into code.
Comment 16 William A. Rowe Jr. 2018-11-07 21:09:21 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.