Bug 46282

Summary:	apache2.2 reverse proxy causing full memory and swap
Product:	Apache httpd-2	Reporter:	Drew Withers <drew>
Component:	mod_proxy	Assignee:	Apache HTTPD Bugs Mailing List <bugs>
Status:	RESOLVED LATER
Severity:	normal	CC:	sf
Priority:	P2	Keywords:	MassUpdate
Version:	2.2.9
Target Milestone:	---
Hardware:	PC
OS:	Linux

Description Drew Withers 2008-11-24 16:44:26 UTC

I'm running Debian Lenny with Apache mpm prefork 2.2.9-10 on x86. I have it set up as a reverse proxy for a few thousand user directories as well as some virtual hosts. When I have reverse proxy enabled for all of these, a single request to my server causes apache to fill up the ram and start swapping until swap is full. This doesn't happen when I disable proxying to the user directories. It also doesn't happen upon loading apache. Only after the first request is processed. It doesn't cause apache to crash but it makes it go very slowly.

Here is a small portion of the config:
ProxyPassMatch ^/~es44(/.*)?$ http://www.et.byu.edu/~es44$1
ProxyPassReverse /~es44 http://www.et.byu.edu/~es44
ProxyPassMatch ^/~bdefig(/.*)?$ http://www.et.byu.edu/~bdefig$1
ProxyPassReverse /~bdefig http://www.et.byu.edu/~bdefig
ProxyPassMatch ^/~rmariana(/.*)?$ http://www.et.byu.edu/~rmariana$1
ProxyPassReverse /~rmariana http://www.et.byu.edu/~rmariana
ProxyPassMatch ^/~scornwel(/.*)?$ http://www.et.byu.edu/~scornwel$1
ProxyPassReverse /~scornwel http://www.et.byu.edu/~scornwel
ProxyPassMatch ^/~gkh3(/.*)?$ http://www.et.byu.edu/~gkh3$1
ProxyPassReverse /~gkh3 http://www.et.byu.edu/~gkh3
ProxyPassMatch ^/~camoo(/.*)?$ http://www.et.byu.edu/~camoo$1
ProxyPassReverse /~camoo http://www.et.byu.edu/~camoo
ProxyPassMatch ^/~karhu(/.*)?$ http://www.et.byu.edu/~karhu$1
ProxyPassReverse /~karhu http://www.et.byu.edu/~karhu
ProxyPassMatch ^/~websterb(/.*)?$ http://www.et.byu.edu/~websterb$1
ProxyPassReverse /~websterb http://www.et.byu.edu/~websterb

There are few thousand of user directories configured in this manner.
There isn't anything special I do for this setup. I enabled proxy and proxy_http. I use the ssl proxy engine and I have ProxyRequests turned off and ProxyVia On

The problem goes away when I remove all the specific entries for user directories and replace them with:
ProxyPass /~ http://www.et.byu.edu/~
ProxyPassReverse /~ http://www.et.byu.edu/~

Comment 1 Stefan Fritsch 2008-12-03 09:11:31 UTC

(In reply to comment #0)
> When I have reverse proxy enabled for all of these, a single
> request to my server causes apache to fill up the ram and start swapping
> until swap is full.

What size do the apache processes have directly after a restart. To what size do they grow after the first request?

Comment 2 Drew Withers 2008-12-03 13:26:07 UTC

> What size do the apache processes have directly after a restart. To what size
> do they grow after the first request?
> 
Here is what they look like when I have all of the proxies turned on:
$ ps -eo rss,vsz,sz,command |grep /usr/sbin/apache2

Immediately after restart:
  RSS    VSZ    SZ COMMAND
110536 150476 37619 /usr/sbin/apache2 -k start
178636 286644 71661 /usr/sbin/apache2 -k start
178636 286644 71661 /usr/sbin/apache2 -k start
178640 286644 71661 /usr/sbin/apache2 -k start
178636 286644 71661 /usr/sbin/apache2 -k start
178636 286644 71661 /usr/sbin/apache2 -k start



After loading one page:
  RSS    VSZ    SZ COMMAND
14824 150476 37619 /usr/sbin/apache2 -k start
14756 286832 71708 /usr/sbin/apache2 -k start
14668 286824 71706 /usr/sbin/apache2 -k start
14332 286644 71661 /usr/sbin/apache2 -k start
127120 286644 71661 /usr/sbin/apache2 -k start
135356 286644 71661 /usr/sbin/apache2 -k start
135312 286644 71661 /usr/sbin/apache2 -k start
135288 286644 71661 /usr/sbin/apache2 -k start
135816 286644 71661 /usr/sbin/apache2 -k start
135380 286644 71661 /usr/sbin/apache2 -k start
135416 286644 71661 /usr/sbin/apache2 -k start




Here is what it looks like when I have it set up with the catchall that I mentioned before

Immediately after restart:
  RSS    VSZ    SZ COMMAND
50056  67824 16956 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55880  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start


After loading one page:
  RSS    VSZ    SZ COMMAND
50056  67824 16956 /usr/sbin/apache2 -k start
56688  82768 20692 /usr/sbin/apache2 -k start
56960  82840 20710 /usr/sbin/apache2 -k start
55880  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start
55876  82632 20658 /usr/sbin/apache2 -k start

Comment 3 Stefan Fritsch 2008-12-14 01:11:46 UTC

This looks like apache uses about 120MB of memory to store a few thousand config directives and the corresponding compiled regular expressions, i.e. on the order of 10K per directive (depending on how many 'a few' is). This doesn't look optimal but is probably acceptable.

However, regardless of the used memory, your configuration will allways be dead slow because apache has to do thousands of regexp matches for every request. Use some more efficient configuration (e.g. with mod_rewrite and RewriteMaps) instead. If you have problems implementing this, ask on some support mailing list.

Comment 4 William A. Rowe Jr. 2018-11-07 21:08:23 UTC

Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.