| Summary: | Segmentation fault in char_buffer_read when reverse proxying SSL | ||
|---|---|---|---|
| Product: | Apache httpd-2 | Reporter: | M. "Alex" Hankins <lxhankins002> |
| Component: | mod_ssl | Assignee: | Apache HTTPD Bugs Mailing List <bugs> |
| Status: | CLOSED FIXED | ||
| Severity: | major | ||
| Priority: | P3 | ||
| Version: | 2.0.50 | ||
| Target Milestone: | --- | ||
| Hardware: | Sun | ||
| OS: | Solaris | ||
| URL: | http://www.2dv.cl/cepechphp/admin/index_login.html | ||
| Attachments: | proposed fix | ||
Does this still happen if you take mod_rewrite out of the setup and use ProxyPass instead? Yes, I still get what seems to be the same segfault (according to gdb) if I replace these config lines: RewriteEngine on RewriteRule /(.*) https://some.eroom6.iis.server.com/$1 [P] with this one: ProxyPass / https://some.eroom6.iis.server.com/ This backtrace is a little strange:
#1 0xfeafef54 in char_buffer_read (buffer=0x1649ac,
in=0x2000 <Address 0x2000 out of bounds>, inl=8192) at ssl_engine_io.c:348
which means either stack corruption by memcpy or in got passed in as a pointer
to address 8192 somehow.
Can you, from gdb against a core dump, check:
up 2 (into ssl_io_input_read)
print *inctx
info locals
print buf
print len
Hi Joe,
I do suffer from the same bug. Attached is the info you requested. If you need
additional testing, please say so. I cleared some info from the contents of
buffer to not disclose critical information. I hope that's ok and doesn't
hinder your debugging process.
Regards, Stephan
Program received signal SIGSEGV, Segmentation fault.
0xfedf060c in memcpy () from /usr/platform/SUNW,UltraAX-i2/lib/libc_psr.so.1
(gdb) where full
#0 0xfedf060c in memcpy ()
from /usr/platform/SUNW,UltraAX-i2/lib/libc_psr.so.1
No symbol table info available.
#1 0x00048060 in char_buffer_read (buffer=0x196d54,
in=0x196d60 "Cet-Cookie: s"...,
at ssl_engine_io.c:348
No locals.
#2 0x00048448 in ssl_io_input_read (inctx=0x196d38,
buf=0x196d60 "Cet-Cookie: s"...,
len=0xffbeafa4) at ssl_engine_io.c:561
wanted = 8192
bytes = 1533728
rc = 8192
#3 0x00048714 in ssl_io_input_getline (inctx=0x196d38,
buf=0x196d60 "Cet-Cookie: s"...,
len=0xffbeafa4) at ssl_engine_io.c:712
pos = 0x173ea0 ""
status = 1666360
tmplen = 1
offset = 1533728
(gdb) up 2
#2 0x00048448 in ssl_io_input_read (inctx=0x196d38,
buf=0x196d60 "Cet-Cookie: s"...,
len=0xffbeafa4) at ssl_engine_io.c:561
561
(gdb) print *inctx
$1 = {ssl = 0x173db8, bio_out = 0x172210, f = 0x198d68, rc = 0,
mode = AP_MODE_GETLINE, block = APR_BLOCK_READ, bb = 0x198d80, cbuf = {
length = 1, value = 0xffffffff <Address 0xffffffff out of bounds>},
pool = 0x16fea8,
buffer = "Cet-Cookie: s"...,
filter_ctx = 0x17e058}
(gdb) info locals
wanted = 8192
bytes = 1533728
rc = 8192
(gdb) print buf
$2 = 0x196d60 "Cet-Cookie: s"...,
(gdb) print len
$3 = (apr_size_t *) 0xffbeafa4
Ah ha, that's crucial info, thanks.
It looks like the cause is: ssl_io_input_read is called with inctx->mode ==
SPECULATIVE (this only normally happens in the proxy IIRC); ssl_io_input_read
calls char_buffer_read, which executes the case where it does:
buffer->value = NULL;
buffer->length = 0;
and ssl_io_input_read then screws up the inctx->cbuf for good.
/* We want to rollback this read. */
inctx->cbuf.value -= bytes;
inctx->cbuf.length += bytes;
cbuf = { length = 1, value = 0xffffffff <Address 0xffffffff out of bounds>},
per your backtrace.
Created attachment 12459 [details]
proposed fix
The patch attached above should fix the segfaults, I'm not yet sure if this is the cleanest or most correct fix. The fix checked in should be equivalent: http://cvs.apache.org/viewcvs.cgi/httpd-2.0/modules/ssl/ssl_engine_io.c?r1=1.125&r2=1.126 This issue has possible security implications; it's been assigned CVE CAN-2004-0751 (cve.mitre.org). Thanks for the report and for the debugging. |
Overview Description: Intermittent segmentation faults occur in char_buffer_read at ssl_engine_io.c:348 when using a RewriteRule to do reverse proxying to an SSL origin server running IIS. Steps to Reproduce: 1) Set up an IIS server using SSL and running eRoom 6. 2) Add the following directives to httpd.conf: Listen 47290 SSLProxyEngine on RewriteEngine on RewriteRule /(.*) https://some.eroom6.iis.server.com/$1 [P] 3) Visit a URL similar to the following: http://reverse.proxy.com:47290/eRoomASP/CookieTest.asp?facility=facility&URL=%2FeRoom%2FFacility%2FRoom%2F0_4242 If that doesn't cause the segfault, click around for a while. (Yes, reverse proxying from non-SSL to SSL is not a good idea, but it keeps the example simpler.) Actual Results: Segmentation fault in error_log: [Thu Jul 15 19:38:36 2004] [notice] child pid 42 exit signal Segmentation fault (11), possible coredump in /usr/local/httpd-2.0.50 Build Date & Platform: 2004-07-14 build on SunOS 5.8 SUNW,UltraAX-i2 Additional Information: Here is a stack trace from gdb: #0 0xfef5060c in memcpy () from /usr/platform/SUNW,UltraAX-i2/lib/libc_psr.so.1 #1 0xfeafef54 in char_buffer_read (buffer=0x1649ac, in=0x2000 <Address 0x2000 out of bounds>, inl=8192) at ssl_engine_io.c:348 #2 0xfeaff388 in ssl_io_input_read (inctx=0x164990, buf=0x1649b8 "Content-Length: 121\r\nCort/Martonia/0_2615\r\nContent-Length: 121\r\nCent-Length: 121\r\nCmeport/Martonia/0_2615\r\nContent-Length: 121\r\nCmeport/Martonia/0_2615\r\nContent-Length: 121\r\nCrtonia/0_2615\r\nContent-Le"..., len=0xffbea8cc) at ssl_engine_io.c:561 #3 0xfeaff624 in ssl_io_input_getline (inctx=0x164990, buf=0x1649b8 "Content-Length: 121\r\nCort/Martonia/0_2615\r\nContent-Length: 121\r\nCent-Length: 121\r\nCmeport/Martonia/0_2615\r\nContent-Length: 121\r\nCmeport/Martonia/0_2615\r\nContent-Length: 121\r\nCrtonia/0_2615\r\nContent-Le"..., len=0xffbea944) at ssl_engine_io.c:712 #4 0xfeb00118 in ssl_io_filter_input (f=0x1669c0, bb=0x158f98, mode=4290685252, block=APR_BLOCK_READ, readbytes=0) at ssl_engine_io.c:1226 #5 0x42978 in ap_get_brigade (next=0x1669c0, bb=0x158f98, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0) at util_filter.c:474 #6 0x4aab4 in net_time_filter (f=0x158e20, b=0x158f98, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0) at core.c:3600 #7 0x42978 in ap_get_brigade (next=0x158e20, bb=0x158f98, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0) at util_filter.c:474 #8 0x43e0c in ap_rgetline_core (s=0xffbeab94, n=8192, read=0xffbeab90, r=0x1671b8, fold=1, bb=0x158f98) at protocol.c:214 #9 0x441a4 in ap_getline (s=0xffbeccd8 "Content-Length", n=8192, r=0x1671b8, fold=1) at protocol.c:478 #10 0xfe8552d4 in ap_proxy_read_headers (r=0x1821d0, rr=0x1671b8, buffer=0xffbeccd8 "Content-Length", size=8192, c=0x1671b8) at proxy_util.c:457 #11 0xfe833014 in ap_proxy_http_process_response (p=0x157960, r=0x1821d0, p_conn=0x157ee8, origin=0x158168, backend=0x157f00, conf=0xf0268, bb=0x157e98, server_portstr=0xffbeed68 ":47290") at proxy_http.c:755 #12 0xfe833ba4 in ap_proxy_http_handler (r=0x1821d0, conf=0xf0268, url=0x158038 "/eRoomASP/CookieTest.asp?facility=memeport&URL=%2FeRoom%2Fmemeport%2FMartonia%2F0_2615", proxyname=0x0, proxyport=60776) at proxy_http.c:1121 #13 0xfe85435c in proxy_run_scheme_handler (r=0x1821d0, conf=0xf0268, url=0x1839ce "https://eroomhost.aaa.bbb.com/eRoomASP/CookieTest.asp?facility=memeport&URL=%2FeRoom%2Fmemeport%2FMartonia%2F0_2615", proxyhost=0x0, proxyport=0) at mod_proxy.c:1113 #14 0xfe852ed8 in proxy_handler (r=0x1821d0) at mod_proxy.c:418 #15 0x359a8 in ap_run_handler (r=0x1821d0) at config.c:151 #16 0x35fa4 in ap_invoke_handler (r=0x1821d0) at config.c:358 #17 0x32c44 in ap_process_request (r=0x1821d0) at http_request.c:246 #18 0x2df14 in ap_process_http_connection (c=0x157a70) at http_core.c:250 #19 0x40090 in ap_run_process_connection (c=0x157a70) at connection.c:42 #20 0x403a4 in ap_process_connection (c=0x157a70, csd=0x157998) at connection.c:175 #21 0x3422c in child_main (child_num_arg=5) at prefork.c:609 #22 0x343ac in make_child (s=0x8edb0, slot=5) at prefork.c:703 #23 0x345fc in perform_idle_server_maintenance (p=0x8c690) at prefork.c:838 #24 0x34a34 in ap_mpm_run (_pconf=0x0, plog=0x63400, s=0x83000) at prefork.c:1039 #25 0x3ad44 in main (argc=3, argv=0xffbef4ac) at main.c:617