[PATCH] Enable HTTP tunneling for streaming data where data is less than the buffering size Synopsis: Optionally disable buffering in mod_proxy We ran into a situation where a command-response protocol was being tunneled through HTTP (2 sockets 1 is an HTTP GET, the other an HTTP POST) that failed when passed through an Apache reverse proxy. When the data passing from the server < AP_MIN_BYTES_TO_WRITE (8000 decimal) then the bucket brigade buffers the data instead of passing it on. The guts of the patch are basically appending a flush bucket after every read(unless there is already an EOS) if the ProxyWriteThrough directive is set to On --------------------------------------------------------- Chris Conti cmconti@mindspring.com chris.conti@xcellenet.com diff -u \ApacheSrc\httpd-2.0.45\modules\proxy\proxy_http.c \ApacheSrc-orig\httpd-2.0.45\modules\proxy\proxy_http.c --- \ApacheSrc\httpd-2.0.45\modules\proxy\proxy_http.c 2003-05-02 16:22:53.000000000 -0400 +++ \ApacheSrc-orig\httpd-2.0.45\modules\proxy\proxy_http.c 2003-02-03 10:31:50.000000000 -0500 @@ -956,18 +956,6 @@ /* signal that we must leave */ finish = TRUE; } - - /* do we need to always send the data? */ - if (conf->write_through && !finish){ - apr_off_t readbytes; - apr_brigade_length(bb, 0, &readbytes); - - - if(0 != readbytes){ - apr_bucket *e = apr_bucket_flush_create(c->bucket_alloc); - APR_BRIGADE_INSERT_TAIL(bb, e); - } - } /* try send what we read */ if (ap_pass_brigade(r->output_filters, bb) != APR_SUCCESS) { diff -u \ApacheSrc\httpd-2.0.45\modules\proxy\mod_proxy.h \ApacheSrc-orig\httpd-2.0.45\modules\proxy\mod_proxy.h --- \ApacheSrc\httpd-2.0.45\modules\proxy\mod_proxy.h 2003-04-29 09:20:12.000000000 -0400 +++ \ApacheSrc-orig\httpd-2.0.45\modules\proxy\mod_proxy.h 2003-02-03 10:31:50.000000000 -0500 @@ -201,8 +201,6 @@ bad_body } badopt; /* how to deal with bad headers */ char badopt_set; - int write_through; - char write_through_set; } proxy_server_conf; diff -u \ApacheSrc\httpd-2.0.45\modules\proxy\mod_proxy.c \ApacheSrc-orig\httpd-2.0.45\modules\proxy\mod_proxy.c --- \ApacheSrc\httpd-2.0.45\modules\proxy\mod_proxy.c 2003-04-29 09:20:12.000000000 -0400 +++ \ApacheSrc-orig\httpd-2.0.45\modules\proxy\mod_proxy.c 2003-02-22 11:38:14.000000000 -0500 @@ -503,8 +503,6 @@ ps->timeout_set = 0; ps->badopt = bad_error; ps->badopt_set = 0; - ps->write_through = 0; - ps->write_through_set = 0; return ps; } @@ -532,7 +530,6 @@ ps->preserve_host = (overrides->preserve_host_set == 0) ? base->preserve_host : overrides->preserve_host; ps->timeout= (overrides->timeout_set == 0) ? base->timeout : overrides->timeout; ps->badopt = (overrides->badopt_set == 0) ? base->badopt : overrides->badopt; - ps->write_through = (overrides->write_through_set == 0) ? base->write_through : overrides->write_through; return ps; } @@ -816,16 +813,6 @@ psf->req_set = 1; return NULL; } -static const char* - set_proxy_writethrough(cmd_parms *parms, void *dummy, int flag) -{ - proxy_server_conf *psf = - ap_get_module_config(parms->server->module_config, &proxy_module); - - psf->write_through = flag; - psf->write_through_set = 1; - return NULL; -} static const char * set_proxy_error_override(cmd_parms *parms, void *dummy, int flag) { @@ -1079,8 +1066,6 @@ "This overrides the server timeout"), AP_INIT_TAKE1("ProxyBadHeader", set_bad_opt, NULL, RSRC_CONF, "How to handle bad header line in response: IsError | Ignore | StartBody"), - AP_INIT_FLAG("ProxyWriteThrough", set_proxy_writethrough, NULL, RSRC_CONF, - "on if the data should be not be buffered"), {NULL} };
add patchavailable keyword.
Your patch looks reversed. You might also want to "attach" it rather than pasting it in, since the line-wrapping makes your patch very hard to apply.
The preferred mechanism would be to AUTOMATICALLY send down a flush bucket if a non-blocking read returns EAGAIN. No configuration directive would be necessary. The simplistic algorithm would be done = 0; while (!done) { rv = ap_get_brigade(APR_NONBLOCK_READ); if (APR_STATUS_IS_EAGAIN(rv)) { pass flush bucket down output filter chain; rv = ap_get_brigade(APR_BLOCK_READ); } if (rv != APR_SUCCESS) { done = 1; break; } } The content-length filter does this, though with a slightly more complicated and optimized algorithm.
In what context do you intend the algorithm to be used? In proxy_http.c in place of the change I proposed? I would think that ordinarily a reverse proxy would want the caching behavior. Only when tunneling dynamic content would you want to disable the caching. In our case, the back end server withholds sending more data until a response is received on a different socket. When I originally was researching this I came across references to people having a similar issue tunneling streaming multimedia.
The logic to hold on to output until we get 8K is not intended to be caching behavior, but instead to send out full packets so that we don't abuse the network. But if we don't pass flush buckets down at the right time, it breaks some applications that need to send output to the client a piece at a time. A flush bucket should be passed down when we've read some output but we find out that another read would block. At this point it is safe to assume that the CGI/origin-server/whatever is not going to generate more output for a while and the output generated so far should be flushed to the client.
This is fixed on HEAD using the method Jeff suggested: http://cvs.apache.org/viewcvs.cgi/httpd-2.0/modules/proxy/proxy_http.c?r1=1.201&r2=1.202
*** Bug 33029 has been marked as a duplicate of this bug. ***
Created attachment 19034 [details] patch for httpd-2.0.59
(In reply to comment #8) > Created an attachment (id=19034) [edit] > patch for httpd-2.0.59 > Guess you provided the reverse patch :-). Furthermore I do not think that this works. The same code does not work in 2.2.x as all non blocking reads are converted to blocking reads somewhere down in the call chain (currently cannot remember where) and thus the conditions in the if clause causing a flush bucket to be added never become true. This is no fault of the proxy code. Either the problem somewhere down the call chain needs to be fixed what seems to be a larger task or we need to use a poll approach here similar to the one currently used in mod_proxy_ajp on the trunk (aka autoflush property of a proxy worker).
After more testing... The patch does not work if Transfer-Encoding is chunked. (See modules/http/http_protocol.c around line 870).
Created attachment 19075 [details] patch for httpd-2.0.59 (works also with TE chunked). Also patch http-protocol to allow the correct to work with TE chunked.
From first glance this looks good. Mind to produce a patch against trunk, such that it can be committed and follow the usual backporting process?
Created attachment 19091 [details] patch for htttpd-trunk patch for htttpd-trunk (06/11/2006).
+1 on the patch
The issue with the chunk filter is a separate issue to the bug in the proxy which was tracked here so please open a new PR if you want that bug tracked.
Backported to 2.2.x as r602679 (http://svn.apache.org/viewvc?rev=602679&view=rev).