The worker MPM does not call apr_setup_signal_thread() until after it runs the child_init hooks for modules. If a module starts a background thread, that thread will occasionally intercept signals that were meant for the main thread. In our case, SIGHUPs sent from the parent process to its workers were intercepted, and we were left with orphaned workers. The documentation for apr_setup_signal_thread() says "Warning: This must be called before any threads are created", so presumably it's incorrect for any module to start a background thread unless it first calls this function anyway. Calling the function inside the child_init hook solves the problem but is somewhat dirty. Steps to Reproduce: 1) Create a module that starts a background thread in child_init using apr_thread_create(), and load it into httpd. The thread should do some sort of long-lived blocking activity, like a select(). 2) Open enough connections to httpd to allow the workers to hit their connection limit and be recycled. Actual Results: Orphaned httpd worker processes will start to pile up. Occasionally some will correctly exit (I haven't investigated to see why), but the majority stick around doing nothing until Apache is stopped. If you attach with GDB and catch SIGHUP, you can see that the signal is interrupting the module's background thread instead of the main worker thread. Expected Results: The workers should correctly exit. GDB should show the main thread being interrupted by SIGHUP. Build Date & Hardware: Apache 2.4.10 (built 2015-05-15) on 64-bit Linux (OpenEmbedded kernel 3.14.37). Additional Builds and Platforms: This ordering (child_init followed by apr_setup_signal_thread) is present in trunk, in both mpm_worker and mpm_event.
We probably need a new hook (mpm_setup_child?), run by each MPM after apr_setup_signal_thread() is called, and where custom threads could be created.