We have been having intermittent Apache crashes on OS X 10.7.5 (Lion Server). These crashes seem to happen at random and we've tried everything to find the cause. We started noticing them more when we changed was MaxRequestsPerChild to 10 to stop memleaks on some scripts we are running. We noticed that the crashes seem to happen when the children are coming to the end of their life and have served their 10 requests. We also notice the segfaults come in waves. For example, we won't notice any for three hours and the all of a sudden they happen every minute for 10 faults and then go quiet again (possible somebody running a vulnerability scanner). The problem may be with PHP or Apache itself. We have also noticed the same crashes with XCache/APC disabled. The crashes seem to always be on "Thread 1" which is kernel/OS side of things most of the time - e.g.: Thread 1 Crashed: 0 libsystem_kernel.dylib 0x00007fff910eb192 __workq_kernreturn + 10 1 libsystem_c.dylib 0x00007fff8fe96594 _pthread_wqthread + 758 2 libsystem_c.dylib 0x00007fff8fe97b85 start_wqthread + 13 Any help, suggestions, tips or bug fixed are very greatly appreciated. Although, please do not suggest upgrading PHP/Apache or OS X - we need a fix for the system in its current state; we're not in a position to upgrade/recompile anything on such a live server so feel free to let us know of patches or fixes (band-aids!) either on the OS side or Apache side. We would love help in finding the cause or what page request creates the crash (however we can't use mod_whatkilledus because we haven't compiled with exception hooks. Any suggested Apache modules and debugging tools are greatly appreciated. Our crash report: <pre> Process: httpd [52830] Path: /usr/sbin/httpd Identifier: httpd Version: ??? (???) Code Type: X86-64 (Native) Parent Process: httpd [86996] Date/Time: 2013-05-15 09:39:36.067 +0100 OS Version: Mac OS X Server 10.7.5 (11G63) Report Version: 9 Crashed Thread: 1 Dispatch queue: com.apple.libdispatch-manager Exception Type: EXC_CRASH (SIGSEGV) Exception Codes: 0x0000000000000000, 0x0000000000000000 Application Specific Information: objc[86996]: garbage collection is OFF Thread 0:: Dispatch queue: com.apple.main-thread 0 libphp5.so 0x0000000103dbd7e2 _zend_hash_add_or_update + 75 1 libphp5.so 0x0000000103d6f5ad zend_symtable_update + 208 2 libphp5.so 0x0000000103d7047b php_register_variable_ex + 1584 3 libphp5.so 0x0000000103b5311d php_sapi_filter + 964 4 libphp5.so 0x0000000103d6a462 sapi_getenv + 99 5 libphp5.so 0x0000000103d74a7d php_security_log + 193 6 libphp5.so 0x0000000103dcccd2 _zend_mm_free_canary_int + 110 7 libphp5.so 0x0000000103d9786e _efree + 52 8 libphp5.so 0x0000000103db07b8 _zval_dtor_func + 56 9 libphp5.so 0x0000000103daa6ce destroy_op_array + 181 10 libphp5.so 0x0000000103dbc712 zend_hash_destroy + 53 11 libphp5.so 0x0000000103daa849 destroy_zend_class + 100 12 libphp5.so 0x0000000103dbc464 zend_hash_apply_deleter + 189 13 libphp5.so 0x0000000103dbc50f zend_hash_reverse_apply + 102 14 libphp5.so 0x0000000103da8dd2 shutdown_executor + 650 15 libphp5.so 0x0000000103db1963 zend_deactivate + 99 16 libphp5.so 0x0000000103d61a4d php_request_shutdown + 583 17 libphp5.so 0x0000000103e2a5d4 php_handler + 1186 18 httpd 0x0000000103755031 ap_run_handler + 97 19 httpd 0x0000000103755bd6 ap_invoke_handler + 422 20 httpd 0x0000000103791616 ap_internal_redirect + 118 21 mod_rewrite.so 0x0000000103a5dc70 handler_redirect + 256 22 httpd 0x0000000103755031 ap_run_handler + 97 23 httpd 0x0000000103755bd6 ap_invoke_handler + 422 24 httpd 0x00000001037909cd ap_process_request + 125 25 httpd 0x000000010378c5c3 ap_process_http_connection + 131 26 httpd 0x000000010376c651 ap_run_process_connection + 97 27 httpd 0x000000010376cc31 ap_process_connection + 113 28 httpd 0x0000000103799c73 child_main + 1699 29 httpd 0x0000000103799eb3 make_child + 435 30 httpd 0x000000010379a23f perform_idle_server_maintenance + 703 31 httpd 0x000000010379a96c ap_mpm_run + 1740 32 httpd 0x0000000103760077 main + 4055 33 httpd 0x0000000103754374 start + 52 Thread 1 Crashed:: Dispatch queue: com.apple.libdispatch-manager 0 libsystem_kernel.dylib 0x00007fff8e2557e6 kevent + 10 1 libdispatch.dylib 0x00007fff89870786 _dispatch_mgr_invoke + 923 2 libdispatch.dylib 0x00007fff8986f316 _dispatch_mgr_thread + 54 Thread 1 crashed with X86 Thread State (64-bit): rax: 0x0000000000000004 rbx: 0x00007fff763a31a0 rcx: 0x00000001432cb528 rdx: 0x0000000000000000 rdi: 0x000000000000001b rsi: 0x0000000000000000 rbp: 0x00000001432cb6a0 rsp: 0x00000001432cb528 r8: 0x0000000000000001 r9: 0x0000000000000000 r10: 0x00000001432cb538 r11: 0xffffff80002dad60 r12: 0x00007fff763a2540 r13: 0x0000000000000000 r14: 0x00007fff763a3230 r15: 0x0000000000000000 rip: 0x00007fff8e2557e6 rfl: 0x0000000000000247 cr2: 0x00007fff763a4310 Logical CPU: 0 External Modification Summary: Calls made by other processes targeting this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by all processes on this machine: task_for_pid: 4267 thread_create: 0 thread_set_state: 0 VM Region Summary: ReadOnly portion of Libraries: Total=143.3M resident=54.1M(38%) swapped_out_or_unallocated=89.2M(62%) Writable regions: Total=96.9M written=30.2M(31%) resident=36.1M(37%) swapped_out=0K(0%) unallocated=60.9M(63%) REGION TYPE VIRTUAL =========== ======= MALLOC 86.3M MALLOC guard page 48K STACK GUARD 56.0M Stack 8712K VM_ALLOCATE 688K __CI_BITMAP 80K __DATA 10.9M __LINKEDIT 51.0M __TEXT 92.3M __UNICODE 544K mapped file 1.0G shared memory 12K =========== ======= TOTAL 1.3G </pre>
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.