The function cgi_read_stdout() in mod_cgi allocates unnecessary amounts of memory. For every call of apr_file_read(), APR_BUCKET_BUFF_SIZE (currently 8000) bytes of memory are allocated, even when apr_file_read() delivers only a fraction of the requested amount of data. In practice, this can lead to hugely inflated httpd processes. The attached patch reduces this problem by allocating only as much memory as necessary.
Created attachment 25431 [details] patch to reduce memory usage of mod_cgi.
Your patch causes all output from the cgi to be copied again, which is not really efficient. Besides, the memory of the bucket should be freed as soon as the data has been sent over the network. I guess what you are really seeing is one of the memory usage issues fixed in r821471 in trunk. You could try if the content_length_filter and chunk_filter parts of r821471 fix your problem. It's http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/chunk_filter.c?r1=821471&r2=821470&pathrev=821471 and the first three hunks of http://svn.apache.org/viewvc/httpd/httpd/trunk/server/protocol.c?r1=821471&r2=821470&pathrev=821471 I haven't checked if it applies cleanly to 2.2.x, though.
The memory usage of mod_cgi is constant in trunk, even if the cgi streams large amounts of data.
2.4.1 is released, closing. If you find additional issues in 2.4.x, please re-open.