Bug 59842 - Undefined symbol ap_get_useragent_host causes server to crash when performing graceful restart.
Summary: Undefined symbol ap_get_useragent_host causes server to crash when performing...
Status: NEEDINFO
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_access_compat (show other bugs)
Version: 2.4.23
Hardware: PC Linux
: P2 major (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-07-11 18:54 UTC by Shawn Bailly
Modified: 2016-07-12 13:09 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Bailly 2016-07-11 18:54:25 UTC
It appears that a change in 2.4.19 in mod_access_compat.c:279, which changes the call of ap_get_remote_host to use ap_get_useragent_host causes an undefined symbol (ap_get_useragent_host) failure when doing a graceful reload of apache2. A full server restart works without issue, but the graceful causes the server to crash.

I first experienced this issue last night after an upgrade to 2.4.20 from 2.4.16 in which the server crashed and did not resume after log rotation. I have also tried to use 2.4.23 which has this issue as well, and looked through other reported bugs and could not find a duplicate.

I have recompiled mod_access_compat.c after changing the line in question back to the pre 2.4.19 changes and have not had any issue with graceful reloading. However, I know that is a short-term fix and there may be a larger issue with as to why the newer call of ap_get_useragent_host is undefined.

Here's the line from my error.log:
apache2: Syntax error on line 140 of /etc/apache2/apache2.conf: Syntax error on line 2 of /etc/apache2/mods-enabled/access_compat.load: Cannot load /usr/lib/apache2/modules/mod_access_compat.so into server: /usr/lib/apache2/modules/mod_access_compat.so: undefined symbol: ap_get_useragent_host

And the mod_access_compat.load file to show there's not really a syntax error there:
# Depends: authn_core
LoadModule access_compat_module /usr/lib/apache2/modules/mod_access_compat.so

Also discovered at least one other person suffering the same issue from this Server Fault article (I replied with my experience and solution): http://serverfault.com/questions/780793/apache2-not-running-after-graceful-restart/789177#789177

Happy to help in any way I can.
Comment 1 Eric Covener 2016-07-11 18:59:32 UTC
(In reply to Shawn Bailly from comment #0)
> It appears that a change in 2.4.19 in mod_access_compat.c:279, which changes
> the call of ap_get_remote_host to use ap_get_useragent_host causes an
> undefined symbol (ap_get_useragent_host) failure when doing a graceful
> reload of apache2. A full server restart works without issue, but the
> graceful causes the server to crash.

It should only happen if you overwrite the module while 2.4.19 is running and try to perform a graceful restart. Your process is 2.4.19 but loading a post-2.4.19 module.  Did it really persist past a single full stop/start of Apache?
Comment 2 Shawn Bailly 2016-07-12 13:09:59 UTC
I don't see my first comment (I replied via email) here so I'll paste it below.
-------------------------------------------------------------------------------
To be clear, the version I upgraded from was 2.4.16 not 2.4.19. I just noticed from looking over the change log that 2.4.19 is where the ap_get_useragent_host change was first introduced.

It's going to be hard to duplicate your suspicion that it was only due to the server not being reloaded because I've already upgraded from 2.4.16 and have worked around the issue, however, on my production servers I have not yet replaced the mod_access_compat with the one that I recompiled. I'll do that some time later, but I'll try a graceful reload first and report back.

However, I do not run from source, therefore I did not compile the new version and replace the files without restarting. The apt-get package, from Ubuntu, does perform a restart any time apache is updated (and it did Saturday night when I updated). So, I think that its a safer bet that even though the server was fully restarted after the update, the graceful still failed. So, this is likely to affect more people moving forward.

To further complicate matters -- because why not -- I went ahead and tried to roll back the updated mod_access_compat.so file on my development server that I had discovered the fix for. That server which was originally updated from 2.4.16 to 2.4.23 via the Ubuntu package manager, did present the same bug (before I applied by work-around). I then replaced the mod_access_compat.so file, restarted, tested graceful reload, and confirmed the updated module worked. But, to fully rule out your suspicion that it was restart related I just now tried to put the package manager version of the mod_access_compat.so back in place, without restarting. Then I performed a graceful restart, which presented no issue. I could not duplicate the original issue with the package manager's mod_access_compat.so file in any way.

Like I said, I'll test the graceful reload again with my unaltered production servers. If it fails I'll try the restart again, then test the graceful reload, if that fails then it definitely is an issue with the updated version. I'll report back.
-----------------------------------------------------------------------------

Last night I tried the graceful reload again and you were correct, it worked with no changes to the module. So, apparently the server had to be shut down and started again completely, not just a restart which occurs during the install process. I'm not sure if that is another issue or not. I've actually encountered something similar when changing out the MPM, a reload or restart would not work. I had to issue a stop, then a start.

Maybe this is something for the package managers to be aware of, but I figured its probably still important for general users who maybe won't expect it to understand what might have happened.

Thanks for help.