Bug 48440

Summary: Segmentation Fault
Product: Apache httpd-2 Reporter: aurabhi <aurabhi>
Component: AllAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED LATER    
Severity: critical Keywords: MassUpdate
Priority: P2    
Version: 2.2.14   
Target Milestone: ---   
Hardware: Sun   
OS: Solaris   

Description aurabhi 2009-12-23 08:49:11 UTC
I have a apache 2.2.14 running Solaris 5.10
Apache is built using SunStudio 12 Update 1. 
On this setup I have a custom module and wlproxy(mod_wl_22.so)
When I request pages which are delivered via wlproxy(mod_wl_22.so) I see segmentation faults. The requests are server properly but at the end of every request I see the following error

[Wed Dec 23 21:08:52 2009] [notice] child pid 11624 exit signal Segmentation fault (11)

Following is what I see when I opened the core in dbx 

bash-3.00# dbx httpd /tmp/apache2-gdb-dump/core
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.7' in your .dbxrc
Reading httpd
core file header read successfully
Reading ld.so.1
Reading libm.so.2
Reading libaprutil-1.so.0.3.9
Reading libexpat.so.0.1.0
Reading libapr-1.so.0.3.9
Reading libuuid.so.1
Reading libsendfile.so.1
Reading librt.so.1
Reading libsocket.so.1
Reading libnsl.so.1
Reading libpthread.so.1
Reading libc.so.1
Reading libaio.so.1
Reading libmd.so.1
Reading libc_psr.so.1
Reading libCrun.so.1
Reading mod_alias.so
Reading mod_wl_22.so
Reading mod_cwmp.so
t@1 (l@1) program terminated by signal SEGV (Segmentation Fault)
0xff0465c8: __pollsys+0x0008:   blu      _cerror        ! 0xfefa34c0
(dbx) where
current thread: t@1
=>[1] __pollsys(0x4, 0x0, 0xffbf9820, 0x0, 0xffbf9820, 0x0), at 0xff0465c8
  [2] _pollsys(0xffbf97b8, 0x0, 0xffbf9820, 0x0, 0x0, 0x0), at 0xff03965c
  [3] _pselect(0xffbf97b8, 0xff0723d0, 0xff0723d0, 0x0, 0xffbf9820, 0x0), at 0xfefe6b10
  [4] _select(0x0, 0x0, 0x0, 0x0, 0xffbf9888, 0xf4240), at 0xfefe6e88
  [5] apr_sleep(0x0, 0xf4240, 0x1, 0x1, 0x0, 0x40000), at 0xff224fe0
  [6] ap_wait_or_timeout(0x11000, 0xffffffff, 0xffbf9a30, 0xa93e8, 0x7, 0xffbf99b4), at 0x4a9e8
  [7] server_main_loop(0x9e5cc, 0x3f, 0x0, 0x9d324, 0x1f, 0x20), at 0x6ba6c
  [8] ap_mpm_run(0x20, 0x9ea40, 0x9d320, 0x0, 0x20, 0x60), at 0x6c840
  [9] main(0x9bb98, 0x9cc00, 0x7c608, 0x9ce1c, 0x9cc00, 0x7c504), at 0x299dc

Help me out please.
Comment 1 Nick Kew 2009-12-23 09:27:16 UTC
Could this be Bug 48030?

Your dbx information is unhelpful: it just shows the parent process.  If it's not the same as PR 48030, you should try to diagnose the role of your third-party module.
Comment 2 aurabhi 2009-12-23 18:59:21 UTC
Is there a way to get a dump from the child which exits due to the SEGV or a
way to force apache to dump core for every SEGV?

Any help would be really gr8. I am totally new to this field. 
Thank you!


(In reply to comment #1)
> Could this be Bug 48030?
> 
> Your dbx information is unhelpful: it just shows the parent process.  If it's
> not the same as PR 48030, you should try to diagnose the role of your
> third-party module.
Comment 4 aurabhi 2009-12-28 02:51:30 UTC
I could get the stack using dbx of a particular thread which crashes. I see that apr_file_read throws the SEGV. 

The problem is the content from the file are read and request gets processed. I get response back on my client but I see this SEGV error always. At times I see that request times out. 

I tried using std file io instead of apr_file io. In which case I see an error in __fseek(); 


(dbx) cont
t@65 (l@65) signal SEGV (no mapping at the fault address) in apr_file_read at 0xff212fcc
0xff212fcc: apr_file_read+0x0020:       ld       [%i0 + 32], %g3
(dbx) where
current thread: t@65
=>[1] apr_file_read(0xffffffff, 0x1c7c38, 0xfad7bb0c, 0x0, 0xbc6, 0x0), at 0xff212fcc
  [2] apr_file_read_full(0xffffffff, 0x1c7c38, 0xbc6, 0xfad7bb88, 0xfad7bb0c, 0x0), at 0xff211be4
  [3] my_input_filter(0x1ba8b0, 0x1bbc98, 0xbc7, 0x0, 0x1c7c38, 0x0), at 0xfeee46bc
  [4] ap_discard_request_body(0x1b2cf0, 0x4, 0x0, 0xc, 0x0, 0x0), at 0x60334
  [5] ap_finalize_request_protocol(0x1b2cf0, 0x1b2cf0, 0xfeee5ab4, 0x14, 0x1258e0, 0x1b2cf0), at 0x32e3c
  [6] my_handler(0xfffffffe, 0x4, 0x0, 0xc, 0x0, 0x0), at 0xfeee3a38
  [7] ap_run_handler(0x1b6050, 0x7, 0x126470, 0x9dc80, 0x12649c, 0x1), at 0x40dc4
  [8] ap_invoke_handler(0x1b6050, 0x0, 0x126544, 0x0, 0x1258e0, 0x0), at 0x413dc
  [9] ap_process_request(0x1b6050, 0x4, 0x0, 0x9cc00, 0x0, 0x1b6050), at 0x5d898
  [10] ap_process_http_connection(0x1ac2d8, 0x1b6050, 0xafde0, 0x9cd98, 0x80000000, 0x9cc00), at 0x5a418
  [11] ap_process_connection(0x1ac2d8, 0x1ac028, 0x1268b8, 0x0, 0x9e4a0, 0x9e400), at 0x490bc
  [12] worker_thread(0x129958, 0x3be, 0x1afff8, 0x1ac2d8, 0x9d340, 0x3e), at 0x6aad0
Comment 5 Ruediger Pluem 2009-12-28 06:50:51 UTC
(In reply to comment #4)
> I could get the stack using dbx of a particular thread which crashes. I see
> that apr_file_read throws the SEGV. 
> 
> The problem is the content from the file are read and request gets processed. I
> get response back on my client but I see this SEGV error always. At times I see
> that request times out. 
> 
> I tried using std file io instead of apr_file io. In which case I see an error
> in __fseek(); 
> 
> 
> (dbx) cont
> t@65 (l@65) signal SEGV (no mapping at the fault address) in apr_file_read at
> 0xff212fcc
> 0xff212fcc: apr_file_read+0x0020:       ld       [%i0 + 32], %g3
> (dbx) where
> current thread: t@65
> =>[1] apr_file_read(0xffffffff, 0x1c7c38, 0xfad7bb0c, 0x0, 0xbc6, 0x0), at
> 0xff212fcc
>   [2] apr_file_read_full(0xffffffff, 0x1c7c38, 0xbc6, 0xfad7bb88, 0xfad7bb0c,
> 0x0), at 0xff211be4
>   [3] my_input_filter(0x1ba8b0, 0x1bbc98, 0xbc7, 0x0, 0x1c7c38, 0x0), at
> 0xfeee46bc

This looks like a bug in my_input_filter in a way that it calls apr_file_read_full with invalid parameters. my_input_filter is no code from httpd. Please remove this filter and let us know if the segmentation fault goes away. If not please provide a stack trace without my_input_filter.
Comment 6 aurabhi 2009-12-28 08:14:54 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > I could get the stack using dbx of a particular thread which crashes. I see
> > that apr_file_read throws the SEGV. 
> > 
> > The problem is the content from the file are read and request gets processed. I
> > get response back on my client but I see this SEGV error always. At times I see
> > that request times out. 
> > 
> > I tried using std file io instead of apr_file io. In which case I see an error
> > in __fseek(); 
> > 
> > 
> > (dbx) cont
> > t@65 (l@65) signal SEGV (no mapping at the fault address) in apr_file_read at
> > 0xff212fcc
> > 0xff212fcc: apr_file_read+0x0020:       ld       [%i0 + 32], %g3
> > (dbx) where
> > current thread: t@65
> > =>[1] apr_file_read(0xffffffff, 0x1c7c38, 0xfad7bb0c, 0x0, 0xbc6, 0x0), at
> > 0xff212fcc
> >   [2] apr_file_read_full(0xffffffff, 0x1c7c38, 0xbc6, 0xfad7bb88, 0xfad7bb0c,
> > 0x0), at 0xff211be4
> >   [3] my_input_filter(0x1ba8b0, 0x1bbc98, 0xbc7, 0x0, 0x1c7c38, 0x0), at
> > 0xfeee46bc
> 
> This looks like a bug in my_input_filter in a way that it calls
> apr_file_read_full with invalid parameters. my_input_filter is no code from
> httpd. Please remove this filter and let us know if the segmentation fault goes
> away. If not please provide a stack trace without my_input_filter.

my_input_filter is a filter that I have added which modifies the BB. I can not take this filter away as this my application requires this to talk to a weblogic apache plugin. 

If the seg fault was where I do apr_file_read_full then request flow should fail at that point but whats happening is the requests complete and the response is sent back to the client. I can see all the debug messages that I have included in the debug log. I have handlers for each of the phases including the log_transaction phase and I see all the handlers being invoked and processed successfully. 

Has this got something to do with https://issues.apache.org/bugzilla/show_bug.cgi?id=48029?
Comment 7 William A. Rowe Jr. 2018-11-07 21:09:42 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.