Bug 40970

Summary: very slow start of server when having big number of vhosts
Product: Apache httpd-2 Reporter: Arkadiusz Miskiewicz <arekm>
Component: CoreAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED DUPLICATE    
Severity: normal    
Priority: P2    
Version: 2.2.3   
Target Milestone: ---   
Hardware: Other   
OS: other   

Description Arkadiusz Miskiewicz 2006-11-14 04:16:44 UTC
Hi.

There is a big problem with apache server when having big number of vhosts. 
Currently I have more than 10000 vhosts defined and when starting httpd it 
stat()ing each DocumentRoot entry.

stat()ing these takes very long time (even several minutes if there is bigger load 
on the server).

Is so many stat() calls really needed for apache? I guess that in config 
verification mode (-t) that would be OK but it's a waste of time for normal 
startup.
Comment 1 Joshua Slive 2006-11-17 16:16:10 UTC
Doesn't directly answer your question, but if you have 10000 vhosts, you should
really be using mod_vhost_alias.

Joshua.
Comment 2 Arkadiusz Miskiewicz 2006-11-17 16:22:17 UTC
I need custom settings per vhost. Also mod_vhost_alias doesn't allow to use some 
external file for vhost->directory mapping itp (like example.com->/home/something/
5323/). In other words - it's too limited.
Comment 3 Joshua Slive 2006-11-18 15:39:00 UTC
Custom settings can be set with <Directory> sections (other than a few
very-specific things like suexec and obviously SSL).  For more complicated
mapping you use either symlinks or mod_rewrite.

Dynamic vhosting doesn't solve all problems, but you'd need a very compelling
reason not to use it with 10000 virtual hosts.
Comment 4 Arkadiusz Miskiewicz 2006-11-18 16:21:48 UTC
Hm, if httpd doesn't do any stat() for <Directory> then I'll guess that some 
mod_rewrite or mod_vhost_alias and 10000 <Directory> entries (yeah, I need to set 
all of these) would save me a lot of stat() calls.

I would love to see module that provides virtual hosting in such way:
/etc/httpd/vhosts/example.org/documentroot symlink to /somewhere/1212
/etc/httpd/vhosts/example.org/vhost.conf - configuration file for virtual host
/etc/httpd/vhosts/www.example.org symlink to example.org
using that way I could easily create virtual hosts and change configuration on the 
fly without any need for apache reload thus avoiding all stat() problems. Also 
reloading takes huuuuge amount of time (30 seconds to several minutes) and when 
that happens server is not responding to new requests. Using runtime vhost.conf 
file that is read and cached on first request to vhost (+ invalidated once per X 
seonds if mtime has changed) the reload problem would be easy to avoid.
Comment 5 Joshua Slive 2006-11-18 20:05:54 UTC
Take it to the users list.  People will be able to help you there if you are
specific about what kinds of config customizations you need to do for the
various vhosts.

I'll leave the original report open because I seem to recall other people
requesting the same thing.  In fact, a quick search reveals that this was fixed
at one point in 1.3.  I'm not sure why it is different in 2.0:
http://mail-archives.apache.org/mod_mbox/httpd-cvs/199906.mbox/%3C19990624163855.13932.qmail@hyperreal.org%3E
Comment 6 Arkadiusz Miskiewicz 2006-12-17 14:09:57 UTC
I've applied such patch to my httpd copy. That directory check was doing more harm 
than help.

--- server/core.c.org   2006-11-22 12:37:15.991248750 +0100
+++ server/core.c       2006-11-22 12:39:30.535657250 +0100
@@ -1129,7 +1129,7 @@
     /* XXX Shouldn't this be relative to ServerRoot ??? */
     if (apr_filepath_merge((char**)&conf->ap_document_root, NULL, arg,
                            APR_FILEPATH_TRUENAME, cmd->pool) != APR_SUCCESS
-        || !ap_is_directory(cmd->pool, arg)) {
+        ) {
         if (cmd->server->is_virtual) {
             ap_log_perror(APLOG_MARK, APLOG_STARTUP, 0,
                           cmd->pool,
Comment 7 Nick Kew 2009-12-27 03:02:02 UTC

*** This bug has been marked as a duplicate of bug 41887 ***