|Summary:||MPM event, stuck process after graceful: no (old gen)|
|Product:||Apache httpd-2||Reporter:||Stephan Schultchen <stephan.schultchen>|
|Component:||mpm_event||Assignee:||Apache HTTPD Bugs Mailing List <bugs>|
Description Stephan Schultchen 2019-02-12 10:33:08 UTC
apachectl graceful fails to kill some processes, which in return keeps logfiles forever open after logrotate. "apachectl fullstatus" says in the "stopping" row that the processes are "no (old gen)"
Comment 1 Yann Ylavic 2019-02-12 13:19:30 UTC
Since there are some changes in MPM event since 2.4.34, can you reproduce with the latest/2.4.38 version?
Comment 2 Stephan Schultchen 2019-02-12 13:55:36 UTC
2.4.38 is not available as a IUS rpm package. i updated some systems to 2.4.37. and check if this solves the issue
Comment 3 Stephan Schultchen 2019-02-13 08:40:10 UTC
the problem still persists with apache httpd 2.4.37 from the IUS repository. the changelog from 2.4.37 seem to not have any changes related to MPM, or anthing else that looks related to log file handling. quite now apache has some processes running since yesterday, that are in the "no (old gen)" stopping status, and these processes have the old, rotated logfiles open.
Comment 4 Yann Ylavic 2019-02-14 12:20:06 UTC
What's the state of old gen processes/threads? Number of requests currently being processed, idle workers, status of the threads (letters S/R/W/K/L/D/C/./G/I/?), full status for "stuck" processes please?
Comment 5 akhil.jaggarwal 2019-05-13 19:00:50 UTC
I believe it is change introduced in 2.4.34 to run cleanups on exit in single process mode: http://svn.apache.org/viewvc/httpd/httpd/trunk/os/unix/unixd.c?view=diff&r1=1830743&r2=1830744&pathrev=1830744 httpd drops privileges to switch to a non-root user if the User directive is defined in httpd.conf. If httpd is required to started as root to bind to well known ports, and if the User directive is defined in the conf, piped logs remain running as root while http d runs as a non-privileged user. This causes graceful restarts on receipt of a SIGTERM to fail, causing both httpd and the piped log processes to remain stuck and not respond to SIGTERMs. A possible fix within apache would be to add code to drop privilges or any sub processes registered with the server (for e.g., the piped log processes) like it is done for httpd. A workaround is to setuid the piped log processes to that of the User in httpd.conf using something like ap_uname2id() so as to make the owner of the piped logs same as the parent httpd process. Additionally, after making the setuid change in our piped log binaries, we've seen core dumps around free() in apr_pool_destroy when cleaning up the apr pool. I'm still investigating these but meanhile any help from the apache team would be well appreciated.