Bug 54101

Summary: Socket backend support for mod_proxy
Product: Apache httpd-2 Reporter: Francesco <framelio>
Component: mod_proxyAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED FIXED    
Severity: enhancement CC: admin, apache_bugzilla, blaise.tarr, diemuzi, erik, glicerinu, lists, m.hofer117, pothi, tobias.pal, tomas, ubunciak
Priority: P2 Keywords: FixedInTrunk, PatchAvailable
Version: 2.4.3   
Target Milestone: ---   
Hardware: All   
OS: Linux   
Attachments: Patch for socket support on mod_proxy
Unix domain socket support for mod_proxy (patch)
Patch for 2.4.x (or 2.4.7)

Description Francesco 2012-11-05 16:10:23 UTC
Created attachment 29549 [details]
Patch for socket support on mod_proxy

Currently mod_proxy and mod_proxy_fcgi doesn't support for a UNIX socket backend.

Speaking about FCGI protocol and PHP, generally many PHP-FPM server configurations are using a socket backend instead of the standard TCP connection.

NGINXx supports it and the mod_fastcgi too with the config:
FastCgiExternalServer /cgi-bin/php-path -socket /path/to/socket/php-fpm.sock

A proposed patch and solution can be found here:
http://www.gossamer-threads.com/lists/apache/dev/416627

The example config format proposed is the following:

ProxyPass fcgi://socket=%2ftmp%2fphp-fpm.sock/local/htdocs/ 

I tested the patch on the version 2.4.3 and it works, even if the patch file should be slightly adapted to the latest changed in the Apache HTTPD code.
Comment 1 Blaise Tarr 2013-02-26 21:43:57 UTC
Created attachment 29995 [details]
Unix domain socket support for mod_proxy (patch)

I updated my patch to resolve a problem on BSD systems. This patch was generated against Apache 2.4.4.

Below is my original announcement on the Apache dev mailing list:

The attached patch adds support for Unix domain sockets (UDS) to 
mod_proxy. This is very beneficial when configuring a reverse proxy 
using mod_proxy_fcgi to connect to PHP-FPM. In some tests with ab 
there was over 20% improvement in the request rate when using a UDS 
instead of TCP. 

Since remote servers in mod_proxy are configured with a URL, I took 
the idea from this page on how to represent a UDS as a URL: 
http://daniel.haxx.se/blog/2008/04/14/http-over-unix-domain-sockets/ 

So the host portion of the URL contains "socket=" followed by the URI 
encoded socket path (upper case characters must be encoded too). Below 
are a couple of examples where PHP-FPM's socket is /tmp/php-fpm.sock 
and the document root is /local/htdocs: 

ProxyPass fcgi://socket=%2ftmp%2fphp-fpm.sock/local/htdocs/ 

RewriteRule (.*\.php) fcgi://socket=\%2ftmp\%2fphp-fpm.sock/local/htdocs/$1 [P,L] 

The following are not contained in the patch: 
* bump MMN due to mod_proxy.h changes 
* assign APLOGNO numbers 
* detection of AF_UNIX support in configure 

Thanks for your consideration, 
Blaise
Comment 2 Graham Leggett 2013-02-26 23:35:46 UTC
Not had a chance to review the patch, but a definite +1 in principle.
Comment 3 DH 2013-02-27 09:55:12 UTC
+1 for stable implementation in the next http 2.4.x releases
Comment 4 Jim Jagielski 2013-03-01 15:52:30 UTC
Will fold into trunk for testing and enhancement. Thx
Comment 5 phpfpm1 2013-03-03 11:21:04 UTC
Would it be possible to fix it to support RewriteRule in <Directory></Directory> block (now RewriteRule only works if not in Directory section, if it is placed in Directory block, then it does not replace %2f to / well (leaves f instead of /)). Now there is no any other way to use it in Directory block, because ProxyPassMatch does not work there too (https://issues.apache.org/bugzilla/show_bug.cgi?id=54616).
Comment 6 luyanfei78 2013-03-14 01:15:57 UTC
There's one problem haven't solved by this patch. The function proxy_fcgi_canon in mod_proxy_fcgi.c will modify proxy url.
fcgi://socket=%2ftmp%2fphp-fpm.sock/local/htdocs/ 
this url will become
fcgi://socket=%2ftmp%2fphp-fpm.sock:8000/local/htdocs/ 
So, it's  important to use nocanon in ProxyPass. If you choose ProxyPassMatch, then your proxy url will always be changed.
Comment 7 ubunciak 2013-03-27 13:38:56 UTC
+1 for stable implementation, waiting for this so long. It's critical to use with php-fpm
Comment 8 phpfpm1 2013-04-25 10:21:12 UTC
Please check https://issues.apache.org/bugzilla/show_bug.cgi?id=54616#c7. It is related to this bug.
Comment 9 ryotakatsuki 2013-09-04 07:53:41 UTC
I see the patch already included in trunk but the part in which the  PROXY_WORKER_MAX_HOSTNAME_SIZE and PROXY_WORKER_MAX_NAME_SIZE were increased is not there (I realized that after get a "ProxyPass worker name (fcgi://uds=%2fsome%2fsocket.sock/tmp/somedirectory too long)"). Was that dropped intentionally?
Comment 10 Peter Boling 2013-09-19 17:16:01 UTC
This will allow Phusion Passenger Standalone to be reverse proxied by Apache as a unix domain socket!  Right now Apache can only reverse proxy over TCP, but Nginx can reverse proxy to a socket, making it the better performer for Ruby apps using this setup, and making me jealous, since I have an all Apache infrastructure.

Any chance of backporting this to 2.2 stable?
Comment 11 Chris Pitchford 2013-10-02 14:06:57 UTC
I second this, 64 bytes is not sufficiently long. It is prohibitively short when, for example, using Amazon AWS long hostnames for back end systems

Have I missed a reason why it needs to be so short? According to wikipedia (and I fully appreciate this is very unprofessional to quote WP)

http://en.wikipedia.org/wiki/Hostname

Each label must be between 1 and 63 characters long,[2] and the entire hostname (including the delimiting dots) has a maximum of 255 characters

Can we assume that the backend can be FQDN not just a single host? 255 characters would seem a minimum for a FQDN. 64 is only sufficient for a single hostname.


(In reply to ryotakatsuki from comment #9)
> I see the patch already included in trunk but the part in which the 
> PROXY_WORKER_MAX_HOSTNAME_SIZE and PROXY_WORKER_MAX_NAME_SIZE were increased
> is not there (I realized that after get a "ProxyPass worker name
> (fcgi://uds=%2fsome%2fsocket.sock/tmp/somedirectory too long)"). Was that
> dropped intentionally?
Comment 12 Marc Zampetti 2013-10-09 20:15:45 UTC
+1 for adding this or similar capability to support PHP-FPM.
Comment 13 me 2013-11-27 01:53:37 UTC
Does the patch work on latest 2.4.7 release?
It hasn't been included, but I can always manually patch it if there are no dependencies.
Comment 14 Sammie S. Taunton 2013-11-30 19:38:41 UTC
(In reply to me from comment #13)
> Does the patch work on latest 2.4.7 release?
> It hasn't been included, but I can always manually patch it if there are no
> dependencies.

Appears the patch does not work with 2.4.7

patching file mod_proxy.h
Hunk #1 succeeded at 236 (offset 4 lines).
Hunk #2 succeeded at 308 (offset 4 lines).
patching file proxy_util.c
Hunk #2 succeeded at 2028 (offset 7 lines).
Hunk #3 succeeded at 2045 (offset 7 lines).
Hunk #4 FAILED at 2156.
Hunk #5 succeeded at 2403 (offset 19 lines).
Hunk #6 succeeded at 2471 (offset 19 lines).
Hunk #7 succeeded at 2607 (offset 19 lines).
1 out of 7 hunks FAILED -- saving rejects to file proxy_util.c.rej
Comment 15 me 2013-12-01 02:22:17 UTC
Damn, guess I need to use mod FastCGI till the next major version release. 
Thankfully someone was kind enough to release a 2.4 compatible version https://github.com/ByteInternet/libapache-mod-fastcgi
Comment 16 iDen 2014-01-03 07:49:46 UTC
Hope this will be fixed in some future minor release of 2.4.x
Very crucial for my production env to switch on apache 2.4, but I'm using PHP-FPM with unix-sockets and definitely won't go with TCP sockets.
Before that's done will stay on apache 2.2 and/or nginx.
Comment 17 Jose 2014-01-20 12:30:54 UTC
Its a pity we had a patch, it was not implemented into development and now it doesn't work anymore.

Stuck in 2.2.. And trying nginx.
Comment 18 Eric Covener 2014-01-20 13:11:32 UTC
(In reply to Jose from comment #17)
> Its a pity we had a patch, it was not implemented into development and now
> it doesn't work anymore.
> 
> Stuck in 2.2.. And trying nginx.

This support has been in trunk/2.5 since early last year.  The patch being considered for backport to 2.4.x is heree:

http://people.apache.org/~jim/patches/uds-2.4.patch

The latest status on it is here:

https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/STATUS

Of course, as it ages, there may be cosmetic breakage of the diff.  Any feedback is welcome.
Comment 19 Yann Ylavic 2014-01-20 13:19:19 UTC
Created attachment 31232 [details]
Patch for 2.4.x (or 2.4.7)

This patch is the same as http://people.apache.org/~jim/patches/uds-2.4.patch (modulo CHANGES/manual/ap_mmn.h) working with current 2.4.x (and 2.4.7).
Comment 20 Vedran Rodic 2014-03-21 15:38:19 UTC
It seems that this patch has landed  in 2.4.8/2.4.9.

But I'm not sure how to use it with ProxyPassMatch

I've tried the example above (fcgi://socket=%2ftmp%2fphp-fpm.sock/local/htdocs/ ) but that didn't work.
Comment 21 Francesco 2014-03-22 15:55:29 UTC
I did a test and the correct syntax for ProxyPassMatch is

ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/path/to/socket.sock|fcgi://localhost/path/to/docroot/

The host localhost doesn't really matter, you can put whatever.
Comment 22 me 2014-03-23 02:29:44 UTC
(In reply to Francesco from comment #21)
> I did a test and the correct syntax for ProxyPassMatch is
> 
> ProxyPassMatch ^/(.*\.php(/.*)?)$
> unix:/path/to/socket.sock|fcgi://localhost/path/to/docroot/
> 
> The host localhost doesn't really matter, you can put whatever.

That works now. Although if I put $1 (as described at http://wiki.apache.org/httpd/PHP-FPM), it fails with a 503.
Comment 23 Eric Covener 2014-07-08 11:41:18 UTC
in 2.4.7
Comment 24 RoyBellingan 2014-10-17 00:43:25 UTC
The proper syntax in my case Apache/2.4.6 (Linux/SUSE)
is unix:///var/run/php-fpm.sock|fcgi://127.0.0.1:9000/srv/www/$1
With a triple slash, otherwise apache complain of ProxyPass URL must be absolute!