| Summary: | CPU consumption due to circular linked list in ap_core_input_filter() | ||
|---|---|---|---|
| Product: | Apache httpd-2 | Reporter: | venkatunix02 |
| Component: | Core | Assignee: | Apache HTTPD Bugs Mailing List <bugs> |
| Status: | RESOLVED LATER | ||
| Severity: | normal | Keywords: | MassUpdate |
| Priority: | P2 | ||
| Version: | 2.2.15 | ||
| Target Milestone: | --- | ||
| Hardware: | HP | ||
| OS: | HP-UX | ||
| Attachments: |
trace
log of top and tusc(syscalls) Mapping between stack trace, source code and itanium assembly code |
||
Created attachment 31228 [details]
trace
Created attachment 31229 [details]
log of top and tusc(syscalls)
Created attachment 31230 [details]
Mapping between stack trace, source code and itanium assembly code
What compiler and APR versions are you using ? If the former is gcc, can you compile with option "-fno-strict-aliasing" and see if it helps ? (In reply to Yann Ylavic from comment #4) > What compiler and APR versions are you using ? > > If the former is gcc, can you compile with option "-fno-strict-aliasing" and > see if it helps ? Thanks for checking this. We use HP C compiler and APR version is 1.4. The attachment 31230 [details] suggests you use APR <= 1.4.2 (without this fix http://svn.apache.org/viewvc?view=revision&revision=1074877). struct apr_bucket_brigade { apr_pool_t *p; struct apr_bucket_list { struct apr_bucket *next; struct apr_bucket *prev; }list; apr_bucket_alloc_t *bucket_alloc; } should be (from APR 1.4.3+) : struct apr_bucket_brigade { apr_pool_t *p; struct apr_bucket_list { struct apr_bucket * volatile next; struct apr_bucket * volatile prev; }list; apr_bucket_alloc_t *bucket_alloc; } Can you upgrade to the latest APR 1.4.8 and see if it helps? Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated. |
We run Apache 2.2.15(+ fixes for certain CVEs) on HP Itanium server with HP-UX. Once in a while, we observe ~100% CPU consumption by the httpd processes. 1. configuration MPM worker configuration: <IfModule mpm_worker_module> ServerLimit 80 StartServers 67 MaxClients 4000 MinSpareThreads 3350 MaxSpareThreads 4000 ThreadsPerChild 50 MaxRequestsPerChild 1200 </IfModule> web server: Reverse proxy 3rd party module: Oracle webgate 2. Logs Please find the attached files: a) stack trace : bt_all b) top and syscall trace (top_syscall_trace) c) Mapping between stack trace, source code and itanium assembly code(trace_srccode_assemblyCode) 3. Trace identified from the core dump generated in that scenario: syscall trace shows that main thread waits for the user space thread that run ap_core_input_filter() trace: Thread 22 (system thread 1947121): #0 0x40000000000977f0:2 in ap_core_input_filter () at core_filters.c:136 #1 0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489 #2 0xc000000000ac76c0:0 in logio_in_filter () at mod_logio.c:129 #3 0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489 #4 0x4000000000074980:0 in ap_rgetline_core () at protocol.c:231 #5 0xc000000004a33580:0 in ap_proxygetline () at mod_proxy_http.c:1658 #6 0xc000000004a338c0:0 in ap_proxy_http_process_response () at mod_proxy_http.c:1733 #7 0xc000000004a35ee0:0 in proxy_http_handler () at mod_proxy_http.c:2434 #8 0xc000000004929a10:0 in proxy_run_scheme_handler () at mod_proxy.c:2675 #9 0xc000000004928b60:0 in proxy_handler () at mod_proxy.c:1024 #10 0x400000000009d2d0:0 in ap_run_handler () #11 0x400000000009e9e0:0 in ap_invoke_handler () at config.c:381 #12 0x40000000000cdec0:0 in ap_process_request () at http_request.c:282 #13 0x40000000000c5a40:0 in ap_process_http_connection () at http_core.c:190 #14 0x40000000000b8d20:0 in ap_process_connection () at connection.c:189 #15 0x40000000000de4c0:0 in process_socket () at worker.c:590 #16 0x40000000000df860:0 in worker_thread () at worker.c:974 #17 0xc000000006f95200:0 in dummy_worker () at threadproc/unix/thread.c:160 #18 0xc00000000013fb20:0 in __pthread_bound_body () at pthreads/pthread.c:4875 Eventhough it shows Line:136, actual loop happens @ line 141: 140 /* ### This is bad. */ 141 BRIGADE_NORMALIZE(ctx->b); Due to compiler optimzation and macro @ line 141, this line no is not shown in trace. As shown in the trace_srccode_assemblyCode attachment, linked list has become circular because address and content of (ctx->b)->list))->next are same (gdb) info register r44 gr44: 0x6000000000777de8 (gdb) x/gx0x6000000000777de8 0x6000000000777de8: 0x6000000000777de8 Prev’s value also contains the same. (gdb) x/gx 0x6000000000774508 0x6000000000774508: 0x6000000000777de8 Please help to resolve this.