Bug 56413

Summary: can not restart httpd on centos 5.10 64bit
Product: Apache httpd-2 Reporter: dick <625953844>
Component: mod_setenvifAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: NEW ---    
Severity: blocker CC: lakshmi, szg0000
Priority: P2    
Version: 2.4.9   
Target Milestone: ---   
Hardware: Other   
OS: Linux   

Description dick 2014-04-15 11:10:15 UTC
/usr/local/webserver/apache/bin/apachectl 
[Tue Apr 15 18:29:09.344674 2014] [core:crit] [pid 28990] AH00102: [Tue Apr 15 18:29:09 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
/usr/local/webserver/apache/bin/apachectl: line 79: 28990 已放弃               $HTTPD "$@"
Comment 1 Lakshmi Mekala 2014-04-30 18:56:05 UTC
I have the same issue and we are on RHEL 5.6. I can install 2.4.7 fine, but 2.4.9 fails. If anyone fixed the issue please let me know.

Thank you
Comment 2 bkiddie 2014-05-01 08:12:44 UTC
I have encountered the same issue with RHEL 5.10. I have tried installing using pcre-8.34 with the same result.
Comment 3 Jan Ehrhardt 2014-05-02 08:36:04 UTC
I have got the same issue on Centos 5. See
http://forum.directadmin.com/showthread.php?t=48569&page=2&p=250879#post250879
Comment 4 Rainer Jung 2014-05-02 09:11:41 UTC
Which version of PCRE is your running httpd actually using?

If you are not sure, you can try to inspect in the following way:

- Start httpd
- Check process id of parent process, say it is 1234
- "grep pcre /proc/1234/maps"
- The output should contain the pcre lib file name that actually gets used
- If that file was installed using an RPM check that RPM for version info.
- If not, you can do a
     strings /lib64/libpcre.so.0.0.1 | egrep -e '[0-9]\.[0-9]'
  (with appropriate change in path and maybe soname). The result contains a version line similar to "6.6 06-Feb-2006". Which one is it?

Even if you installed a new version of pcre, please double check if your httpd actually uses it.

Regards,

Rainer
Comment 5 Jan Ehrhardt 2014-05-02 12:47:30 UTC
I have just downgraded our production server to 5.4.7:
http://forum.directadmin.com/showthread.php?t=48569&page=3&p=250903#post250903

But I will try the upgrade once again on the dev server. In the mean time:

bash-3.2# strings /lib64/libpcre.so.0.0.1 | egrep -e '[0-9]\.[0-9]'
GLIBC_2.4
GLIBC_2.3
GLIBC_2.3.4
GLIBC_2.2.5
6.6 06-Feb-2006
Comment 6 Jan Ehrhardt 2014-05-02 14:30:57 UTC
After the upgrade of our dev server to Apache 2.4.9:

bash-3.2# grep pcre /proc/5027/maps
7f3a3e96e000-7f3a3e9ab000 r-xp 00000000 ca:01 4654444                    /usr/local/lib/libpcre.so.0.0.1
7f3a3e9ab000-7f3a3ebaa000 ---p 0003d000 ca:01 4654444                    /usr/local/lib/libpcre.so.0.0.1
7f3a3ebaa000-7f3a3ebab000 rw-p 0003c000 ca:01 4654444                    /usr/local/lib/libpcre.so.0.0.1
bash-3.2# strings /usr/local/lib/libpcre.so.0.0.1 | egrep -e '[0-9]\.[0-9]'
GLIBC_2.3
GLIBC_2.2.5
8.20 2011-10-21

When I try to build PHP this happens:

Sorry, I cannot run apxs.  Possible reasons follow:

1. Perl is not installed
2. apxs was not found. Try to pass the path using --with-apxs2=/path/to/apxs
3. Apache was not built using --enable-so (the apxs usage page is displayed)

The output of /usr/sbin/apxs follows:
[Fri May 02 16:29:21.208426 2014] [core:crit] [pid 9049] AH00102: [Fri May 02 16:29:21 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
sh: line 1:  9049 Aborted                 /usr/sbin/httpd -l
apxs:Error: Sorry, no shared object support for Apache.
apxs:Error: available under your platform. Make sure.
apxs:Error: the Apache module mod_so is compiled into.
apxs:Error: your server binary `/usr/sbin/httpd'..
configure: error: Aborting

*** There was an error while trying to configure php. Check the configure file
Comment 7 Rainer Jung 2014-05-02 19:24:51 UTC
Can you confirm, that you also have the problem that the OP reported, ie. the error shows up when you do a graceful restart?

Or do you "only" have the problem of building php?

In the latter case, is /usr/sbin/apxs the apxs that was installed with your httpd 2.4.9 and /usr/sbin/httpd your 2.4.9 httpd binary?

If so, do you get the same error when simply running "/usr/sbin/httpd -l"?

When running that and before setting "export LD_LIBRARY_PATH=/usr/local/lib" which contains your newer PCRE?

Sorry about the many questions, but I'm trying to understand the situation.
Comment 8 Rainer Jung 2014-05-02 20:16:35 UTC
And one more question: is mod_setenvif compiled as a shared object file (mod_setenvif.so) and loaded via LoadModule, or compiled in statically?
Comment 9 Lakshmi Mekala 2014-05-02 21:20:12 UTC
The apache 2.4.9 compiles and builds fine. After the install when I do :

apachectl restart
[Fri May 02 15:11:42.789241 2014] [core:crit] [pid 16074:tid 47630748573680] AH00102: [Fri May 02 15:11:42 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
in/apachectl: line 79: 16074 Aborted                 $HTTPD -k $ARGV


It won't even kill the old process and no errors in the error_log either. 

I have tried the compilation with new pcre-8.35 also and I have forced to use this pcre-8.35 when I compile the apache:  --with-pcre=/usr/local/pcre-8.35/pcre-config

I have also tried killing the old PIDs and just doing apachectl start, I get same error. 

We don't have PHP in our environment, so I think the issue is unrelated to PHP version.
Comment 10 Jan Ehrhardt 2014-05-03 10:06:29 UTC
(In reply to Rainer Jung from comment #7)
> Can you confirm, that you also have the problem that the OP reported, ie.
> the error shows up when you do a graceful restart?

No, it restarts fine.

> Or do you "only" have the problem of building php?

Looks like it.

> In the latter case, is /usr/sbin/apxs the apxs that was installed with your
> httpd 2.4.9 and /usr/sbin/httpd your 2.4.9 httpd binary?

This was the clue. It was an old apxs from Sep 23, 2012. Apparently, 'make install' does not install a new apxs. When I copied the new support/apxs to /usr/sbin, my troubles were gone. I will report this to the Directadmin folks.

BTW: mod_setenvif was compiled in statically.
Comment 11 Jan Ehrhardt 2014-05-03 10:35:49 UTC
(In reply to Lakshmi Mekala from comment #9)
> The apache 2.4.9 compiles and builds fine.

Does it install the new apxs? If not, try chmodding support/apxs to 755 and copying it to /usr/sbin.
Comment 12 Lakshmi Mekala 2014-05-03 23:56:52 UTC
(In reply to Jan Ehrhardt from comment #11)
> (In reply to Lakshmi Mekala from comment #9)
> > The apache 2.4.9 compiles and builds fine.
> 
> Does it install the new apxs? If not, try chmodding support/apxs to 755 and
> copying it to /usr/sbin.

Yes it is installing new apxs. However I tried restarting apache after I copied support/apxs as you said and sill I see same error:

[Sat May 03 17:45:14.069265 2014] [core:crit] [pid 32379:tid 47901599981552] AH00102: [Sat May 03 17:45:14 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
bin/apachectl: line 79: 32379 Aborted                 $HTTPD -k $ARGV


Do you think setenvif module is corrupt in that httpd-2.4.9.tar.gz package itself? Because when I comment the LoadModule setenvif_module part the apachectl restart works until these lines:

<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>


Ofcourse the restart will fail because I have commented the LoadModule part.

Also do you know what is changed in 2.4.9 related to this setenvif module from version 2.4.7. Because the same httpd.conf file works in 2.4.7.

Thank you
Comment 13 Jan Ehrhardt 2014-05-06 15:10:01 UTC
(In reply to Jan Ehrhardt from comment #10)
> > In the latter case, is /usr/sbin/apxs the apxs that was installed with your
> > httpd 2.4.9 and /usr/sbin/httpd your 2.4.9 httpd binary?
> 
> This was the clue. It was an old apxs from Sep 23, 2012. Apparently, 'make
> install' does not install a new apxs. When I copied the new support/apxs to
> /usr/sbin, my troubles were gone. I will report this to the Directadmin
> folks.

Hmmm. The bug recurred with the Directadmin build script:
http://forum.directadmin.com/showthread.php?t=48569&page=3&p=251053#post251053
Comment 14 msg 2014-05-06 21:58:14 UTC
I reproduced the problem, please check the following:

-bash-3.2# cat tst
#!/bin/sh
LD_LIBRARY_PATH=/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
httpd -v

-bash-3.2# sh tst
[Tue May 06 23:57:29.273420 2014] [core:crit] [pid 30983] AH00102: [Tue May 06 23:57:29 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
tst: line 4: 30983 Aborted                 httpd -v
Comment 15 msg 2014-05-06 22:35:39 UTC
The problem is that with the mentioned LD_LIBRARY_PATH the following PCRE module gets loaded:
libpcre.so.0 => /lib64/libpcre.so.0
libz.so.1 => /lib64/libz.so.1

While it should be:
libpcre.so.0 => /usr/local/lib/libpcre.so.0
libz.so.1 => /usr/local/lib/libz.so.1

So it might be that there were some changes in required version of PCRE in Apache 2.4.9. And the issue is PCRE-related.
Comment 16 Lakshmi Mekala 2014-05-07 16:08:15 UTC
(In reply to msg from comment #15)
> The problem is that with the mentioned LD_LIBRARY_PATH the following PCRE
> module gets loaded:
> libpcre.so.0 => /lib64/libpcre.so.0
> libz.so.1 => /lib64/libz.so.1
> 
> While it should be:
> libpcre.so.0 => /usr/local/lib/libpcre.so.0
> libz.so.1 => /usr/local/lib/libz.so.1
> 
> So it might be that there were some changes in required version of PCRE in
> Apache 2.4.9. And the issue is PCRE-related.

So we just have to wait on new PCRE release then? I have tried both pcre-8.34 and 8.35 and I see same issue.
Comment 17 Benji Taylor 2014-07-22 18:48:28 UTC
I am following the bug for this over on PCRE they are pointing back to a file in the apache source: 

mod_setenvif.c


Here is the bug over on PCRE:

http://bugs.exim.org/show_bug.cgi?id=1487


I am having the same issues listed in the above comments which is preventing us from releasing anything higher than 2.4.7 for RHEL5.
Comment 18 Ruediger Pluem 2014-07-22 19:08:38 UTC
Does the following patch fix your issue?

Index: util_pcre.c
===================================================================
--- util_pcre.c (revision 1612204)
+++ util_pcre.c (working copy)
@@ -125,7 +125,7 @@
     const char *errorptr;
     int erroffset;
     int errcode = 0;
-    int options = PCRE_DUPNAMES;
+    int options = 0;

     if ((cflags & AP_REG_ICASE) != 0)
         options |= PCRE_CASELESS;
Comment 19 Benji Taylor 2014-07-23 01:04:34 UTC
I just tested the above patch:

Index: util_pcre.c
===================================================================
--- util_pcre.c (revision 1612204)
+++ util_pcre.c (working copy)
@@ -125,7 +125,7 @@
     const char *errorptr;
     int erroffset;
     int errcode = 0;
-    int options = PCRE_DUPNAMES;
+    int options = 0;

     if ((cflags & AP_REG_ICASE) != 0)
         options |= PCRE_CASELESS;

This does fix the issue and corrects any of the above mentioned pcre issues. 2.4.10 starts without issue now.
Comment 20 Ruediger Pluem 2014-07-23 07:22:15 UTC
(In reply to Benji Taylor from comment #19)
> I just tested the above patch:
> 
> Index: util_pcre.c
> ===================================================================
> --- util_pcre.c (revision 1612204)
> +++ util_pcre.c (working copy)
> @@ -125,7 +125,7 @@
>      const char *errorptr;
>      int erroffset;
>      int errcode = 0;
> -    int options = PCRE_DUPNAMES;
> +    int options = 0;
> 
>      if ((cflags & AP_REG_ICASE) != 0)
>          options |= PCRE_CASELESS;
> 
> This does fix the issue and corrects any of the above mentioned pcre issues.
> 2.4.10 starts without issue now.

This means that you compile against a recent version of PCRE, but when you start Apache the RedHat provided PCRE is used. So you need to have a look at your LD_LIBRARY_PATH settings.
Comment 21 Lakshmi Mekala 2014-07-23 18:50:50 UTC
How do we specify current pcre while compiling apache? I have used 
--with-pcre:/usr/local/pcre-8.35

I get the same error after installing 2.4.10 as well:

[Wed Jul 23 12:48:25.293419 2014] [core:crit] [pid 30025:tid 47707166380016] AH00102: [Wed Jul 23 12:48:25 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed
Comment 22 Benji Taylor 2014-07-23 19:02:08 UTC
If you apply this patch when you compile:

Index: util_pcre.c
===================================================================
--- util_pcre.c (revision 1612204)
+++ util_pcre.c (working copy)
@@ -125,7 +125,7 @@
     const char *errorptr;
     int erroffset;
     int errcode = 0;
-    int options = PCRE_DUPNAMES;
+    int options = 0;

     if ((cflags & AP_REG_ICASE) != 0)
         options |= PCRE_CASELESS;

It will fix the issue with:

[Wed Jul 23 12:48:25.293419 2014] [core:crit] [pid 30025:tid 47707166380016] AH00102: [Wed Jul 23 12:48:25 2014] file mod_setenvif.c, line 637, assertion "is_header_regex_regex != NULL" failed

I have checked both versions of PCRE and I have tried the build both ways with using Redhat PCRE and just grabbing PCRE source. I still get the above error in either case unless I manually apply the patch from above. I also verified that the installed apache 2.4 in both cases were using the same version of PCRE that it was compiled against.