|Summary:||Apache 2 crashes with Acept Mutex error when stressed with 20 users on HP-UX 11.00|
|Product:||Apache httpd-2||Reporter:||Arthur Luimes <arthur.luimes>|
|Component:||Core||Assignee:||Apache HTTPD Bugs Mailing List <bugs>|
Description Arthur Luimes 2003-08-18 14:17:25 UTC
During a loadtest with 20 concurrent users, Apache 2.0.47 (using all default config values) crashes on HP-UX 11.00. The error messages are: [Mon Aug 18 15:35:06 2003] [emerg] (28)No space left on device: couldn't grab the accept mutex [Mon Aug 18 15:35:06 2003] [alert] Child 2349 returned a Fatal error... Apache is exiting! [Mon Aug 18 15:35:06 2003] [emerg] (36)Identifier removed: couldn't grab the accept mutex [Mon Aug 18 15:35:06 2003] [emerg] (36)Identifier removed: couldn't grab the accept mutex [Mon Aug 18 15:35:06 2003] [emerg] (36)Identifier removed: couldn't grab the accept mutex ... etc.... There is plenty of space on all devices: this could not be the issue. I've tried adjusting some HP paramateres as in the HP thread: http://forums.itrc.hp.com/cm/QuestionAnswer/0,,0xf91e36e69499d611abdb0090277a778 c,00.html All to no avail. My system is now an easy target for DOS attacks. What to do? P.S. I've load-tested the sever with 10 concurrent users and find no problems.
Comment 1 Jeff Trawick 2003-08-18 14:27:58 UTC
What kind of accept mutex are you using? To determine this: 1st see if there is an AcceptMutex directive in your configuration file if not, the output of "apachectl -V" tells what is the compiled-in default I think it will say -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE which means that SysV semaphores are used for cross-process locks and pthread mutexes are used for intra-process locks meanwhile, ENOSPC (no space left on device) is an error associated with SysV semaphores, and it has nothing to do with available filesystem space Assuming you're really using SysV semaphores: a) the problem may be some OS tuning issue... increase any system tuning associated with SysV sems until the problem goes away b) if you can't get it tuned properly, switch to a different mutex mechanism, such as fcntl, by putting something like this in your config file: AcceptMutex fcntl Note that fcntl mutexes have their own issues. For starters, if your server root is on a remote filesystem, you'll need to use the LockFile directive.
Comment 2 Arthur Luimes 2003-08-18 15:24:13 UTC
Jeff, The setting AcceptMutex fcntl solved the issue! Thanks a lot!! What kind of issues you mentioned below can I expect with fcntl?
Comment 3 Jeff Trawick 2003-08-19 11:20:37 UTC
What issues to expect with fcntl? I have relatively little HP-UX specific knowledge. Some of the issues that have come up on other platforms include: fcntl may not perform as well as SysV semaphore or cross-process pthread locks fcntl may have tuning requirements also, and if you add additional child processes to the server to handle increased load you may suddenly hit an fcntl-related tuning issue BTW, somebody reported the same problem a couple of days earlier (PR 22484). I'll change the closing status of this one to a dup of that one.