Bug 43308

Summary: Persistent backend connections not supported for RewriteRule [P]
Product: Apache httpd-2 Reporter: Axel-Stephane Smorgrav <asmorgrav>
Component: mod_proxyAssignee: Apache HTTPD Bugs Mailing List <bugs>
Severity: normal CC: antoine.prevosto, asmorgrav, takashi.asfbugzilla
Priority: P3 Keywords: MassUpdate, PatchAvailable
Version: 2.2.6   
Target Milestone: ---   
Hardware: All   
OS: All   
Bug Depends on: 43472    
Bug Blocks:    
Attachments: Patch against httpd 2.2.4
Configuration and debug log illustrating the creation of new workers
Patch in Unified format
Patch against httpd 2.2.4
Patch against
Patch against 2.2.8

Description Axel-Stephane Smorgrav 2007-09-04 23:27:45 UTC
In Apache 2.2.x a worker, and the associated backend connection pool, is created
for each ProxyPass directive. In that case the backend connections will by
default be persistent (reusable).

Another mechanism exists for proxying requests to a backend: RewriteRule with
the [P] flag. Unless the URL resulting from the rewriting process matches an
existing worker (created because of the presence of a RewriteRule), the
rewritten request will not be able to benefit from a persistent connection to
the backend. In that case the default worker (*) will be used. The connections
in the pool of the default worker are not persistent.

I therefore suggest to modify ap_proxy_pre_request() in order to create new
workers as needed in the event of requests that do not match an existing worker.
Comment 1 Axel-Stephane Smorgrav 2007-09-04 23:34:28 UTC
Created attachment 20771 [details]
Patch against httpd 2.2.4

The attached patch adds new workers on the fly as new backends for which no
worker exists are identified. This can happen if a RewriteRule proxies a
request (with the [P] flag), and the backend URL does not match any of the
ProxyPass' second argument (URL).

1. New workers are created for scheme://address/. The URL path is ignored. One
might want to add a new configuration directive for specifying the number of
path elements to include in the worker name.

2. The request for which the worker is created is served using the default
worker. The connection pool for the new worker is then only populated on the
first request matching the new worker (the 2nd request), and an existing
connection may only be reused from the following request on.

3. There is no way to tune the connection pooling of the workers created

4. One might want to add a new configuration directive for (de-)activating this

Regarding the implementation details, being a novice to Apache development, I
look forward to your comments. However

i) PROXY_COPY_CONF_PARAMS should probably be moved to mod_proxy.h to avoid
duplication of code

ii) I allocate a copy of the URL on the stack rather than out of the pool since
as far as I could see, it is duplicated in ap_proxy_add_worker() anyway

iii) Someone skilled in the art of Apache httpd development could probably come
up with a way that the newly created worker could be used immediately instead
of having to wait for the next request.
Comment 2 Axel-Stephane Smorgrav 2007-09-05 01:20:44 UTC
Created attachment 20772 [details]
Configuration and debug log illustrating the creation of new workers

The attached file contains the debug log tracing a few requests made against a
patched server.
Comment 3 Paul Querna 2007-09-10 18:52:30 UTC
Can you provide the patch in 'diff -u' format?
Comment 4 Axel-Stephane Smorgrav 2007-09-13 01:54:01 UTC
Created attachment 20808 [details]
Patch in Unified format

Here's the same patch again, this time in diff -u format
Comment 5 Axel-Stephane Smorgrav 2007-09-16 23:52:21 UTC
In order to address certain concerns about possible DoS attacks or potential
resource consumption, rather than a configuration directive for de-activating
this mechanism, I suggest adding a new configuration directive ProxyMaxAddtlWorkers.

It will take a single numeric argument which is the maximum number of workers
that can be implicitly created. The default value would be 0 - zero -
effectively disabling the creation of additional workers.

Apart from limiting the number of workers created by the above described
mechanism, an equivalent number of additional scoreboard entries should also be
allocated during the configuration phase in order to avoid starving client
Comment 6 Axel-Stephane Smorgrav 2007-09-17 02:09:13 UTC
Created attachment 20830 [details]
Patch against httpd 2.2.4

In addition to creating workers implicitly, this new patch implements a new
configuration directive ProxyMaxAddtlWorkers which defaults to 0. This
directive limits the number of workers that can be implicitly created during
request processing and an equivalent number of additional scoreboard entries
are allocated at startup.

Another advantage over the previous patch is that the newly created worker is
used immediately - no need to wait for the next request.

The macro PROXY_COPY_CONF_PARAMS has been moved from mod_proxy.c to mod_proxy.h
because it is also used in proxy_util.c.
Comment 7 Axel-Stephane Smorgrav 2007-09-17 02:29:10 UTC
Created attachment 20831 [details]
Patch against 

The scoreboard entries are not really allocated, but proxy_lb_workers is
incremented by the value of ProxyMaxAddtlWorkers.
Comment 8 Takashi Sato 2007-09-28 23:45:03 UTC
I've read mod_proxy.c and proxy_util.c and found a hackful workaround.
Using ProxyPass to create dummy workers.
Rewriting applied, then uses the dummy ProxyPass's worker.


my server conf:
RewriteEngine On
RewriteRule ^/([^.]+)\.php$ http://localhost:20000/test/$1.php [P]
ProxyPassReverse / http://localhost:20000/test/

ProxyPass /keepalive-dummy/ http://localhost:20000/
Comment 9 Axel-Stephane Smorgrav 2007-10-01 10:26:16 UTC
(In reply to comment #8)
> I've read mod_proxy.c and proxy_util.c and found a hackful workaround.
> Using ProxyPass to create dummy workers.
> Rewriting applied, then uses the dummy ProxyPass's worker.

Yes indeed, but it supposes that you code the exhaustive list of workers into
the configuration. If your backend server list is in a RewriteMap where you can
add and delete workers without restarting Apache, or if you use any other means
to maintain the list of backend servers, this approach does not work.
Comment 10 Axel-Stephane Smorgrav 2008-03-28 02:51:33 UTC
Created attachment 21731 [details]
Patch against 2.2.8
Comment 11 Axel-Stephane Smorgrav 2010-04-28 03:13:22 UTC
We have had this patch in production with several versions of Apache 2.2 including 2.2.14 without having encountered any problems.
Comment 12 William A. Rowe Jr. 2018-11-07 21:08:53 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.