Bug 63231 - Failing OSCP staple blocks httpd server
Summary: Failing OSCP staple blocks httpd server
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_ssl (show other bugs)
Version: 2.4.38
Hardware: PC All
: P2 major (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-04 20:20 UTC by Chris Daley
Modified: 2019-03-10 01:24 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Daley 2019-03-04 20:20:58 UTC
Running an Apache web server out of an IBM Softlayer server farm in Dallas. Evey so often, the Apache hpptd child process restarts (as it should) then stops responding to page requests when it cannot access OSCP stapling. 

The latest was this morning (all times MST) between 2:31 am and 3:01 am. Here's a sample from the log file...

[Mon Mar 04 02:31:15.367980 2019] [mpm_winnt:notice] [pid 18232:tid 532] AH00418: Parent: Created child process 18324
[Mon Mar 04 02:31:32.649236 2019] [mpm_winnt:notice] [pid 18324:tid 3236] AH00354: Child: Starting 1024 worker threads.
[Mon Mar 04 02:31:35.385082 2019] [ssl:error] [pid 18324:tid 24368] (OS 10060)A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.  : [client 207.46.13.24:5884] AH01977: failed reading line from OCSP server
[Mon Mar 04 02:31:35.385082 2019] [ssl:error] [pid 18324:tid 24368] [client 207.46.13.24:5884] AH01980: bad response from OCSP server: (none)
[Mon Mar 04 02:31:35.385082 2019] [ssl:error] [pid 18324:tid 24368] AH01941: stapling_renew_response: responder error

The 3 [ssl:error] lines keep repeating over and over (I have over 40 sites hosted on my server including lodesys.com and k12irc.org). 

Apache is restarted automatically every 5 minutes when not responding. The errors persisted until 3:01 am when the OSCP server appears to have started responding again and no more problems.

Have a ticket in with LetsEncrypt to see what's going on on their end, but would expect that Apache httpd would not lock up when the OCSP server stops responding.

Here's my Apache config lines...

SSLUseStapling          on
SSLStaplingResponderTimeout 2
SSLStaplingReturnResponderErrors off
SSLStaplingFakeTryLater off

SSLStaplingCache "shmcb:${SRVROOT}/logs/ssl_stapling(128000)"
SSLStaplingStandardCacheTimeout 86400

SSLSessionCache  "shmcb:${SRVROOT}/logs/ssl_scache(512000)"
SSLSessionCacheTimeout  300


This issue has happened intermittently over the last week or two. I am looking at the option of just turning off SSLUseStapling as I need the sites up and running 24/7.

Would be nice to get this fixed. Any ideas would be appreciated. 

This has been reported previously in Bug 61818. Using the current ApacheLounge build.
Comment 1 Chris Daley 2019-03-10 01:24:03 UTC
Additional information...

Saw that on Windows, Apache was terminating, stopping the httpd Apache 2.4 service.

Turning SSLUseStapling off "solved" the problem and Apache is again stable. Obviously, not the preferred solution.

Up until about 2 weeks ago, Lets Encrypt local OSCP server had been stable. Something changed (a denial of service attack?) and Apache was intermittently not able to get OCSP responses, causing Apache to fail rather than ignore the fetch errors. (Logs available upon request).