| Summary: | pcre symbol issues for apache modules | ||
|---|---|---|---|
| Product: | Apache httpd-2 | Reporter: | Andres Salomon <dilinger> |
| Component: | Core | Assignee: | Apache HTTPD Bugs Mailing List <bugs> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | alan.schmitt, ben, devin, devincar, dmeranda, jefhen, le-apache, maxime, ogunden, rich |
| Priority: | P3 | Keywords: | PatchAvailable |
| Version: | 2.0-HEAD | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Attachments: |
patch to link against external libpcre
patch to rename internal pcre functions Amended version of patch by Andres Salomon, with typo fixed by Paul Argentoff more intelligent patch to link against PCRE |
||
Created attachment 10733 [details]
patch to link against external libpcre
Created attachment 10734 [details]
patch to rename internal pcre functions
*** Bug 26088 has been marked as a duplicate of this bug. *** hi all, this problem's been around for awhile ... for at least a couple of years! http://groups.google.com/groups?hl=en&lr=&ie=UTF -8&scoring=d&q=pcre+apache+%22multiple+definitions+of+symbol%22&btnG=Search it keeps getting reported primarily to the php boards, and repeatedly referred *back* as an Apache bug. bottom line: ###################################################################### Andres' patch (http://nagoya.apache.org/bugzilla/showattachment.cgi?attach_id=10734) seems to fix the problem. can this get integrated into HEAD & the 2.0.50 RELEASE? ###################################################################### on OSX, for quite awhile, i've been able to get around it by buildg httpd with its internal/included pcre39, building an external build of pcre39, and linking php against the external lib via the LDFLACGS & CPPFLAGS however, with recent (for me, since yesterday) httpd-head, building php --with-apxs aagain causes conflicts with apache's included pcre. any inclusion/combination of any other (later) version, OR the inclusion of the php config param "--with-pcre-regex=..." results in 'make' errors/failures. reading thru Andres' patch submission here, i tried the 'replace-namespace' patch (link above). per Andres, it seems to do the trick for Debian ... as for me, with patch applied in OSX, i'm now able to successfully build httpd w/ internal pcre -- now in its own namespace -- and build php500 --with-pcre against an external build of pcre45. here _my_ 'recipe' for success ... three 'pieces': (1) pcre 45 (2) httpd 2.0.50 (there's a few other issues i'm having with latest HEAD, so i dropped back to 2.0.50 for this) (3) php 500 release also: % uname -v Darwin Kernel Version 7.5.0: Thu Aug 5 19:26:16 PDT 2004; root:xnu/xnu-517.7.21.obj~3/ RELEASE_PPC % glibtool --version ltmain.sh (GNU libtool) 1.5.8 (1.1220.2.117 2004/08/04 14:12:05) % automake --version automake (GNU automake) 1.9 % autoconf --version autoconf (GNU Autoconf) 2.59 % gcc --version gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1666) % /usr/local/ssl/bin/openssl version OpenSSL 0.9.7d 17 Mar 2004 ############################################################ pcre-4.5 wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-4.5.tar.gz gnutar zxf pcre-4.5.tar.gz cd /usr/ports/pcre-4.5 unsetenv CFLAGS CPPFLAGS CXX CXXFLAGS LDFLAGS LDDLFLAGS LD_PREBIND LINGUAS LC_ALL LANG glibtoolize --force --copy && aclocal && autoconf ./configure \ --prefix=/usr/local/pcre45 \ --enable-shared --disable-static make make install ############################################################ httpd-2.0.50 wget ftp://apache.secsup.org/pub/apache/dist/httpd/httpd-2.0.50.tar.gz gnutar zxf httpd-2.0.50.tar.gz cd /usr/ports/httpd-2.0.50 curl -f -L -o patch.pcre.txt "http://nagoya.apache.org/bugzilla/showattachment.cgi?attach_id=10734" patch -p2 < patch.pcre.txt unsetenv CFLAGS CPPFLAGS CXX CXXFLAGS LDFLAGS LDDLFLAGS LD_PREBIND LC_ALL LANG LINGUAS ./buildconf # enable linking against dlcompat's libdl ============================================================= (EDITOR) /usr/ports/httpd-2.0.50/srclib/apr/configure @10433 --- enable_dlopen=no +++ enable_dlopen=yes enable_win32_dll=no ============================================================= ./configure \ --with-mpm=worker \ --enable-layout=Darwin --with-port=80 \ --enable-mods-shared=all --enable-so --disable-static \ --sysconfdir=/etc/apache2 \ --enable-dav --enable-dav-fs --enable-dav-lock \ --enable-ssl --with-ssl=/usr/local/ssl \ --disable-suexec \ --with-z \ --enable-cgi \ --enable-proxy \ --enable-proxy-connect \ --enable-proxy-ftp \ --enable-proxy-http \ --enable-logio \ --enable-authn-dbm --enable-authz-dbm \ --with-imap # these lib links won't pick up from cmd line ... do it manually for now ============================================================= (EDITOR) /usr/ports/httpd-2.0.50/build/config_vars.mk EXTRA_CFLAGS = -g -O2 EXTRA_CXXFLAGS = --- EXTRA_LDFLAGS = -L/usr/local/lib -L/usr/local/ssl/lib +++ EXTRA_LDFLAGS = -bind_at_load -L/usr/local/lib -L/usr/local/ssl/lib --- EXTRA_LIBS = -lssl -lcrypto +++ EXTRA_LIBS = -lssl -lcrypto -lz -ldl -lexpat ============================================================= make make install ################################################################# php-5.0.0 wget http://us2.php.net/distributions/php-5.0.0.tar.gz gnutar zxf php-5.0.0.tar.gz unsetenv CFLAGS CPPFLAGS CXX CXXFLAGS LDFLAGS LDDLFLAGS LD_PREBIND EXTRA_LDFLAGS EXTRA_LIBS LC_ALL LANG LINGUAS ;\ setenv LC_ALL C ;\ setenv LANG C ;\ setenv CPPFLAGS "-I/usr/local/ssl/include -I/usr/local/pcre45/include -I/usr/local/include" ;\ setenv LDFLAGS "-bind_at_load -L/usr/local/ssl/lib -lssl -lcrypto -L/usr/local/pcre45/lib -lpcre" ============================================================= (EDITOR) /usr/ports/php-5.0.0/configure.in --- AC_PROG_RANLIB +++ AC_PROG_LIBTOOL ============================================================= cd /usr/ports/php-5.0.0 autoconf ranlib /usr/local/pgsql/lib/libpq.a ./configure \ --disable-debug \ --prefix=/usr --with-layout=PHP \ --with-config-file-path=/etc/php5 --sysconfdir=/etc/php5 \ --enable-shared --disable-static \ --libdir=/System/Library/PHP \ --includedir=/usr/include \ --mandir=/usr/local/man \ --localstatedir=/var/php4 \ --with-db4=/usr \ --with-pgsql=/usr/local/pgsql --without-mysql \ --enable-cli --with-pear=/System/Library/PHP \ --disable-cgi \ --with-apxs2=/usr/sbin/apxs \ --disable-dmalloc \ --with-tsrm-pthreads \ --enable-shmop --enable-sockets \ --enable-inline-optimization \ --enable-xml --enable-libxml --with-libxml-dir=/usr \ --with-java=/Library/Java/Home \ --with-openssl=/usr/local/ssl --with-openssl-dir=/usr/local/ssl \ --with-zlib --with-zlib-dir=/usr \ --with-imap=/usr/local/imap --enable-mailparse \ --with-imap-ssl=/usr/local/ssl \ --with-mcrypt --with-mhash \ --with-gmp=/usr/local \ --with-gd=/usr/local/gd \ --with-png-dir=/usr/local \ --with-jpeg-dir=/usr/local \ --with-tiff-dir=/usr/local \ --enable-magic-quotes \ --enable-calendar \ --disable-mbstring \ --with-kerberos=/usr \ --with-freetype-dir=/usr/X11R6 \ --with-xpm-dir=/usr/X11R6 \ --enable-exif \ --enable-ftp \ --enable-bcmath \ --with-pcre-regex=/usr/local/pcre45 make make install cheers, richard *** Bug 23952 has been marked as a duplicate of this bug. *** This bug is really important to fix. I can't get my mod_caml module working with Apache 2.0 until it's fixed. This prevents a significant userbase from migrating to Apache 2. My duplicate bug report contains some other information: Bug 23952 I just had this problem as well, trying to install a Caml based wiki. If I add the "LoadModule caml_module modules/mod_caml.so" to my httpd.conf file, apache just keeps segfaulting. Unfortunately this is on a server where we need apache2, so this means that until this fix is integrated, I cannot use mod_caml (and every other mod that need a recent pcre library, it seems). Adding PatchAvailable keyword just building httpd2.0.51 ... it seems this bug is STILL not resolved. there's a demonstrated, working patch (cref: Andreas, below) that fixed the issue in 2.0.50, and (seemingly) applies to 2.0.51 as well ... is there a particular reason why this known/reported/fixed issue is NOT being integrated? thx, richard There are some subtle issues with this: 1) with an httpd built to use an external pcre, the include/pcre.h which gets installed does not match the pcre which is actually used. It's possible that some third-party modules will depend on this. 2) Renaming both the reg* and pcre_* functions is a binary compatibility issue. hi,
i'm back again!
i'm building httpd 2.0.52 + php 5.0.2 on OSX 10.3.5
the ONLY way i'm able to get httpd & php to 'play nice' together -- avoiding later/'downstream' pcre
conflicts -- is to make sure that they're built using the SAME version of pcre.
there are two options to that end ...
(a)
build httpd w/ INTERNAL pcre (v3.9)
build external pcre3.9
build php 502 against pcre3.9/external
(b)
apply Andreas' patch to httpd (*luckily* the patch still works against 2.0.52 ...)
build external pcre5.0
build httpd 2.0.52 against pcre5.0/external
build php 5.0.2 against pcre5.0/external
(a), of course, locks PHP into the use of pcre3.9 ... which is simply not acceptable/tenable for a variety
of other apps that need >= v4.5
so, (b) is really the only functional option here ...
acknowledging 'possible subleties' from Joe's reference above, the php issue is neither subtle, nor
simply possible ... others as well (cref above) are noting problems with, e.g., mod_caml that requires a
more recent pcre.
what need to be done to get this solution addressed/integrated? even ASSIGNED would be a good start!
this has been an unaddressed issue problem for QUITE awhile ...
richard
Agree with the previous comment. mod_caml cannot use anything except an external, modern PCRE, so somehow linking it against Apache's ancient version of PCRE is a non-starter, even if it were possible. This bug is a showstopper for a variety of important modules, so it should be fixed (particularly, since there's even a patch to do it). Created attachment 13377 [details]
Amended version of patch by Andres Salomon, with typo fixed by Paul Argentoff
The following notes were sent to me by Paul Argentoff to accompany the preceeding patch. - - - - - - This is patch by Andres Salomon <dilinger@voxel.net>, with typo fixed by Paul Argentoff <argentoff@rtelekom.ru> This patch fixes problems seen by apache modules attempting to run regex stuff, and getting apache's internal pcre implementation. It contains two parts; the first (the autoconf stuff) checks if libpcre is installed on the host system. If it is, that pcre is used to build apache2. If it's not, apache2 compiles its own pcre, and statically links against it. The second part of this patch forces pcre to be used. Apache, by default, calls the posix regex functions (reg{exec,comp,free,error}) internally. If glibc was sane, this would be fine. Unfortunately, it's not. So, when apache calls these regex functions, it's anyones guess as to whether it will use glibc's regex functions, or the pcre regex functions. The glibc regex functions segfault (within glibc) on my system. So, to ensure sanity, we force use of pcre regex functions. I need to do a bit more testing w/ apache's internal pcre, but for debian's purposes (pcre in /usr), it works fine. Pleeeeeeeeeeeeease incorporate this fix? I'll bake you cookies! Ok, I've created a new patch that should supercede both other patches. This patch checks for regex support via libc, and will use that if found. If the libc doesn't support POSIX.2 regex functions, it will then check for an external pcre; if found, it links against that. If it doesn't find that, it finally falls back to compiling and linking against its own internal pcre library. This patch is also significantly smaller than my other two patches, as it doesn't require any widespread symbol changes. I've tested compilation with and without regex.h and pcre-dev. Testers whose libc specifically lacks regex support are requested (all testers welcome, of course :) I'll follow up w/ fixes as necessary. We won't use any POSIX regex stuff. The PCRE is used on purpose... Created attachment 13661 [details]
more intelligent patch to link against PCRE
Patch against apache 2.0.52; use libc's regex functions, falling back to
external pcre, and finally building internal pcre only if absolutely necessary.
(In reply to comment #17) > We won't use any POSIX regex stuff. The PCRE is used on purpose... Why not? What specifically is broken regarding the libc-provided POSIX regex stuff? And, why not test for breakage, instead of unconditionally using PCRE when it's probably not even necessary? Consistency among the various installations and way more power. In that case, instead of using the POSIX functions, we should probably just be using the PCRE functions... That's a good point and I personally agree. This should be discussed on dev@ anyway. The best would be with a patch backhand :) More than that, httpd *cannot* use a real POSIX regex implementation rather than PCRE: even the default config actually uses some of the Perl extensions to the regex grammar (the "(?:" stuff IIRC?). This is all now done on the trunk based on Andres' patches - thanks a lot! Again, it can't really be backported to 2.0, unfortunately, because it changes the module interface. hi,
> Again, it can't really be backported to 2.0, unfortunately, because it changes the module interface.
so iiuc, then, we've two options at this point:
(1) STABLE 2.0.x release source + Andres' patches (assuming/hoping they continue to apply ...)
(2) UNSTABLE cvs source, w/ functionality included.
i.e., no STABLE solution until 2.1 release ...
am i correct?
thanks for the update!
richard
(In reply to comment #25) I just tried the most recent patch with Apache 2.0.53 and make failed: libtool: link: cannot find the library `/Users/me/Desktop/webserver/httpd-2.0.53/srclib/pcre/ libpcre.la' make[2]: *** [htpasswd] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all-recursive] Error 1 (In reply to comment #25) > > Again, it can't really be backported to 2.0, unfortunately, because it changes > > the module interface. > > so iiuc, then, we've two options at this point: Yes, that's correct. If anyone else is following this, the 2.1.3 alpha is out now, and it solved my problems. *** Bug 28781 has been marked as a duplicate of this bug. *** I have applied most recent Andres' patch (id=13661) against most recent current
stable Apache - 2.0.54. Patched sources compile cleanly, but at run-time Apaches
crashes/segfaults immediately after parsing any regex in config file.
Here are some details: in my system this patch selects glibc's implementation of
regex. If there are not regexes in config file, then apache starts successfully.
If there is regex in config file, it compiles without problem. Using gdb,
(gdb) break regcomp
(gdb) r
(gdb) bt
#0 0x403bf852 in regcomp () from /lib/libc.so.6
#1 0x08071714 in ap_pregcomp (p=0x809b0a8, pattern=0x80d3bd0 "^\\.ht",
cflags=135085008) at util.c:268
#2 0x0807ca4e in filesection (cmd=0xbffff990, mconfig=0x80d3bd0, arg=0x80d3bc1
"") at core.c:1837
#3 0x0806b5f1 in invoke_cmd (cmd=0x808e2e8, parms=0xbffff990,
mconfig=0x80b59f0, args=0x80d2b90 "~ \"^\\.ht\">")
at config.c:797
#4 0x0806bf06 in ap_walk_config (current=0x80d2b70, parms=0xbffff990,
section_vector=0x80b58e0) at config.c:1060
#5 0x0806cd2f in ap_process_config_tree (s=0x80d3bd8, conftree=0x80d3bd0,
p=0x809b0a8, ptemp=0x80d3bd0)
at config.c:1643
#6 0x0806fa6b in main (argc=5, argv=0xbffffab4) at main.c:539
But, shortly, segfaults occurs:
(gdb) cont
Continuing.
Program received signal SIGSEGV, Segmentation fault.
trie_node_alloc (p=0x809b0a8, parent=0x809b518, c=95 '_') at util_filter.c:113
113 if (c == parent->children[i].c) {
(gdb) bt
#0 trie_node_alloc (p=0x809b0a8, parent=0x809b518, c=95 '_') at util_filter.c:113
#1 0x080766a3 in register_filter (name=0x809b518 "core_in", filter_func=
{out_func = 0x807ed90 <core_input_filter>, in_func = 0x807ed90
<core_input_filter>}, filter_init=0, ftype=0,
reg_filter_set=0x809b518) at util_filter.c:217
#2 0x08080611 in register_hooks (p=0x809b0a8) at core.c:4509
#3 0x0806fb4f in main (argc=5, argv=0xbffffab4) at main.c:578
This should be fixed in apache-2.1; Joe Orton incorported my patch into it, plus added some additional fixes. If you're going to use 2.0, don't use my patches (I'm not keeping them up-to-date); use Debian's. Adam Conrad backported Joe's changes for 2.0, and they're being kept up-to-date. ftp://ftp.debian.org/debian/pool/main/a/apache2/ apt-get source apache2, or grab the patch out of the .diff.gz. |
Currently, apache2 includes a full version of pcre-3.9 in its sources; this is built during compilation, and statically linked against. The symbols from this library, are exported to apache modules. This is undesirable for some apache modules; for example, php. Php links against libpcre on the system; however, it will happily pick up the pcre symbols in the apache binary before the symbols in the pcre shared library. This breaks php functions such as pcre_match(), which expect a relatively new version of pcre to be used (3.9 doesn't cut it; it always fails w/ that). There are a number of solutions, here. I attempted upgrading the pcre included in apache; however, I had no way to know if the various win32 hacks introduced to the pcre build system would still work. I opted to not bother, since I have no way to test such work except for on a unix system. An alternate solution is to check for pcre on the build system, and link against that instead of building pcre for apache. This works, but introduces other issues. With apache2 looking for the posix regex functions in shared libraries, it ends up using the glibc implementations for reg{comp,exec,free,error} instead of the pcre implementations of these function. On my system, the glibc implementations segfault somewhere inside glibc; it's much safer to stick w/ the pcre versions. So, aside from the autoconf-ery that's necessary to do the pcre system checks, there are some additional code changes necessary to make sure that the pcre functions are called instead of the glibc functions. This is mainly some ugly code duplication. The patch that implements this change is here: <http://sloth.voxel.net/~dilinger/020-external_pcre.patch> A third solution is to simply change the internal pcre library to have its own namespace. This ensures that apache uses a pcre that it's been fully tested with, while apache modules don't use them in place of a system-supplied pcre. What I've done in the following patch is to prefix the posix regex functions in pcre with ap_pcreposix, and to prefix the other pcre regex functions with ap_pcre. The interface that apache is _suppose to_ export to modules has not changed. That patch is here: <http://sloth.voxel.net/~dilinger/021-pcre_mangle_symbols.patch> I prefer the 021-pcre_mangle_symbols.patch patch, even though it's larger; it ensures stability, at the cost of some code duplication. The 020-external_pcre.patch patch isn't as clean as I'd hoped, and for the added complexity, not much memory is saved. Anyways, links to both patches are supplied so that you folks can decide between the preferred method. Please consider applying either one of my patches, or an equivalent patch, so that pcre issues in external apache modules are solved.