From a post on the Apache httpd-dev mailing list: Hi, The "ProxyErrorOverride On" setting is correctly catching the errors from the (reverse) proxied server. Only, it overrides too much IMHO. Right now it overrides anything that's not in the 2xx range, but I think it should allow also the 3xx range for redirects etc. A commonly used "trick" is to set a cookie with a 302 header so the browser gets redirected to the page which "needs" the cookie. When using ProxyErrorOverride, mod_proxy_http sets its own headers and the cookie is lost. The attached patch check not only for ap_is_HTTP_SUCCESS but also for ap_is_HTTP_REDIRECT which should solve the problem. Thanks, Bart van der Schans I can second this claim, as we also see it with our applications, most notably when attempting to reverse proxy the login process of SAP E-Recruiting, IBM WebSphere Portal Server, and Roller (http://rollerweblogger.org/page/project). I have attached the patch proposed by the original poster. Please see the list thread for some further discussion of this patch
Created attachment 18043 [details] Bart's proposed patch Here are some comments made on this patch on the httpd-dev list by Ruediger Pluem along with Bart's replies: Pl�m wrote: > >> -----Urspr�ngliche Nachricht----- >> Von: Bart van der Schans >> >> Hi, >> >> The "ProxyErrorOverride On" setting is correctly catching the errors >> from the (reverse) proxied server. Only, it overrides too much IMHO. >> Right now it overrides anything that's not in the 2xx range, >> but I think >> it should allow also the 3xx range for redirects etc. > > I had a quick look into this and noticed the following: > > 1. It may make sense to add ap_is_HTTP_INFO to this also. Yes, that shounds like a good idea. > 2. ProxyErrorOverride is currently only honoured by mod_proxy_http, > mod_proxy_ajp ignores it. Is this intended? I don't have much experience with ajp, but being able to set a custom error is a good idea I think. > 3. This is a change in behaviour for people who use customized redirect > pages for browsers that do not support redirects (are there any?) Wouldn't that change from currently broken to working? > 4. 304 not modified responses from the backend are currently not supported > without this patch. I didn't actually tested that. Regards, Bart
Have you tried to set a custom error document? Can you check if ErrorDocument 301 /dummy.html ErrorDocument 302 /dummy.html ErrorDocument 303 /dummy.html fixes your problem? Of course dummy.html must be a file in your document root.
Finally able to work on this bug again. Tested this behavior using version 2.2.3 and it looks like the issue is now resolved. Cookies are correctly set on 302 redirect requests with ProxyErrorOverride On. I suspect changes elsewhere cleaned up this issue since the bug was first reported.
NOT FIXED in vanilla 2.2.3. I'm using mod_proxy_http to reverse proxy a resin application server. Cookies and most headers set by resin in a 302 response are dropped by 2.2.3 when proxyerroroverride is on. What follows are transcripts with our internal 2.2.3 server; the first transcript is with proxyerroroverride on in the specified virtual host config; the second with proxyerroroverride off. Transcript #1, proxyerroroverride on [user@HOST ~]$ telnet HOST 80 Trying 172.20.17.48... Connected to HOST (172.20.17.48). Escape character is '^]'. POST /ec/login.htm HTTP/1.1 Host: HOST Cookie: JSESSIONID=A6534nqtHTaUTMwp-q Content-Type: application/x-www-form-urlencoded Content-Length: 33 userName=z10000&password=password HTTP/1.1 302 Found Date: Fri, 29 Dec 2006 17:56:23 GMT Location: http://HOST/ec/postLogin.htm Content-Length: 234 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://HOST/ec/postLogin.htm">here</a>.</p> </body></html> Connection closed by foreign host. Transcript #2, proxyerroroverride off [user@HOST ~]$ telnet HOST 80 Trying 172.20.17.48... Connected to HOST (172.20.17.48). Escape character is '^]'. POST /ec/login.htm HTTP/1.1 Host: HOST Cookie: JSESSIONID=A6534nqtHTaUTMwp-q Content-Type: application/x-www-form-urlencoded Content-Length: 33 userName=z10000&password=password HTTP/1.1 302 Found Date: Fri, 29 Dec 2006 17:57:44 GMT Server: Resin/3.0.14 Pragma: No-cache Expires: Thu, 01 Jan 1970 00:00:00 GMT Cache-Control: no-cache Cache-Control: no-store Content-Language: en-US Location: http://HOST/ec/postLogin.htm Content-Length: 88 Set-Cookie: userInfo=VARIOUSSTUFF; domain=HOST; path=/ Set-Cookie: XXXX=w0NIFrSTDuo250TP4oXq13pk9C1Rlt9Q; domain=HOST; path=/ Content-Type: text/html; charset=UTF-8 The URL has moved <a href="http://HOST/ec/postLogin.htm">here</a> Connection closed by foreign host.
Update: we applied Bart's proposed patch, with line numbers modified to suit 2.2.3, and it solved the problem for us. So with proxyerroroverride On and patched 2.2.3, we are able to pass cookies back from a resin app server serving 302 responses over http. Very good!
*** Bug 41601 has been marked as a duplicate of this bug. ***
I can confirm this is indeed NOT FIXED in httpd 2.2.4. I had thought it was based on some tests I did earlier, but today we had application who displayed the exact same symptoms as others have reported...when issuing a 302 redirect with ProxyErrorOverride On, the Set-Cookie header was lost. I used the patch from Bug ID 41601 submitted by Stuart Children as this was updated for 2.2.4. I can confirm that applying this patch fixed my problem...could this patch please be applied to both trunk and/or 2.2?
I don't think this can reasonably be described as a bug: rather it's documented behaviour of ProxyErrorOverride. We can't apply the suggested patch, because it breaks that. *You* can of course apply the patch yourself, since it's the behaviour you want. However, it's a reasonable enhancement request, to provide an option to preserve *headers* from the backend while substituting a local response *body*. Simple solution: if you want your cookies from the backend, don't override them.
(In reply to comment #8) > I don't think this can reasonably be described as a bug: rather it's > documented behaviour of ProxyErrorOverride. Huh? http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxyerroroverride [quote] This directive is useful for reverse-proxy setups, where you want to have a common look and feel on the error pages seen by the end user. This also allows for included files (via mod_include's SSI) to get the error code and act accordingly (default behavior would display the error page of the proxied server, turning this on shows the SSI Error message). [/quote] Since when are redirects considered errors? This bug is a *regression* in behaviour from 2.0.x. > However, it's a reasonable enhancement request, to provide an option to > preserve *headers* from the backend while substituting a local response > *body*. We (well, I) don't want that. We want the entire proxied request to be left alone, because there's nothing wrong with it! Don't replace the body with the ErrorDocument, don't remove the headers. > Simple solution: if you want your cookies from the backend, don't override > them. I'm not...
Clearly the documentation is not explicit enough here to define how the implementation should act; I'd certainly agree that not treating 3xx response as "errors" for the purposes of ProxyErrorOverride would be the obvious default. Are you sure this is a regression since vanilla 2.0.x Stuart? AFAICT the 2.0.x code will override errors for any non-2xx response too.
(In reply to comment #10) > Clearly the documentation is not explicit enough here to define how the > implementation should act; I'd certainly agree that not treating 3xx response > as "errors" for the purposes of ProxyErrorOverride would be the obvious > default. Yes, granted it does give a definition of "error pages" - but as you seem to agree, most people would not take that to include redirects (or 304 responses come to that). > Are you sure this is a regression since vanilla 2.0.x Stuart? AFAICT the > 2.0.x code will override errors for any non-2xx response too. Granted we do apply patches and our own modules to the server I saw this on (when just upgrading the httpd version and looking for broken things), but I'm pretty sure none of those would affect this area of behaviour. However, I have confirmed that (see below). The 2.0.x code is actually pretty confused, and potentially broken in other ways. Within ap_proxy_http_process_response we see: * if we are overriding the errors, we can't put the content * of the page into the brigade */ if ( (conf->error_override ==0) || r->status < 400 ) { Which would be correct in my book (though better expressed using the macros). But then later: if ( conf->error_override ) { /* the code above this checks for 'OK' which is what the hook expects */ if ( r->status == HTTP_OK ) return OK; else { int status = r->status; r->status = HTTP_OK; /* Discard body, if one is expected */ if ((status > 199) && /* not any 1xx response */ (status != HTTP_NO_CONTENT) && /* not 204 */ (status != HTTP_RESET_CONTENT) && /* not 205 */ (status != HTTP_NOT_MODIFIED)) { /* not 304 */ ap_discard_request_body(rp); } return status; } } else return OK; which would seem to indicate that only 200 would ever be considered a non-error response (even more wrong I hope you'll agree)! However, there is something more going on - which I've not had time to follow through/debug as yet. Proof is in what an actual vanilla server does, so here goes: Built Apache (2.2.4 and 2.0.58) with: ./configure --prefix=/tmp/httpd-X.X.X --with-mpm=prefork --enable-so --enable-mods-shared='rewrite expires info deflate speling headers unique-id proxy asis' with: gcc (GCC) 4.1.1 20070105 (Red Hat 4.1.1-51) on: Linux gnl05024.int.gnl 2.6.19-1.2895.fc6 #1 SMP Wed Jan 10 19:28:18 EST 2007 i686 i686 i386 GNU/Linux On one server (I guess being fair you'd put this somewhere independent - but I tried it both way rounds), create a CGI which does this: $ curl --get --verbose http://localhost:2058/cgi-bin/redirect * About to connect() to localhost port 2058 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2058 > GET /cgi-bin/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2058 > Accept: */* > < HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:07:36 GMT < Server: Apache/2.0.58 (Unix) < X-My-Secret-Header: moo < Location: http://httpd.apache.org/ < Content-Length: 283 < Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://httpd.apache.org/">here</a>.</p> <hr> <address>Apache/2.0.58 (Unix) Server at localhost Port 2058</address> </body></html> * Connection #0 to host localhost left intact * Closing connection #0 OK, so we've got a 302 response, with a custom header in there. Now take both our 2.2 and 2.0 servers with respective vanilla configs and add: ProxyPass /rproxy http://localhost:2058/cgi-bin ProxyPassReverse /rproxy http://localhost:2058/cgi-bin ProxyErrorOverride On and restart. Now we can make the request to the CGI above but going through each reverse proxy. Firstly, on the 2.0.58 server: $ curl --get --verbose http://localhost:2058/rproxy/redirect * About to connect() to localhost port 2058 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2058 > GET /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2058 > Accept: */* > < HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:16:54 GMT < Server: Apache/2.2.4 (Unix) < X-My-Secret-Header: moo < Location: http://httpd.apache.org/ < Content-Length: 208 < Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://httpd.apache.org/">here</a>.</p> </body></html> * Connection #0 to host localhost left intact * Closing connection #0 Correct HTTP status and custom header is present. Now on the 2.2.4 server: $ curl --get --verbose http://localhost:2204/rproxy/redirect * About to connect() to localhost port 2204 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2204 > GET /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2204 > Accept: */* > < HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:17:40 GMT < Location: http://httpd.apache.org/ < Content-Length: 208 < Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://httpd.apache.org/">here</a>.</p> </body></html> * Connection #0 to host localhost left intact * Closing connection #0 We've lost the headers. QED. Interestingly, making HEAD requests we see behaviour the other way around: $ curl --head --verbose http://localhost:2058/rproxy/redirect * About to connect() to localhost port 2058 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2058 > HEAD /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2058 > Accept: */* > < HTTP/1.1 302 Found HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:18:11 GMT Date: Thu, 15 Feb 2007 18:18:11 GMT < Location: http://httpd.apache.org/ Location: http://httpd.apache.org/ < Content-Type: text/html; charset=iso-8859-1 Content-Type: text/html; charset=iso-8859-1 * Connection #0 to host localhost left intact * Closing connection #0 $ curl --head --verbose http://localhost:2204/rproxy/redirect * About to connect() to localhost port 2204 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2204 > HEAD /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2204 > Accept: */* > < HTTP/1.1 302 Found HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:18:52 GMT Date: Thu, 15 Feb 2007 18:18:52 GMT < Server: Apache/2.2.4 (Unix) Server: Apache/2.2.4 (Unix) < X-My-Secret-Header: moo X-My-Secret-Header: moo < Location: http://httpd.apache.org/ Location: http://httpd.apache.org/ < Content-Type: text/html; charset=iso-8859-1 Content-Type: text/html; charset=iso-8859-1 * Connection #0 to host localhost left intact * Closing connection #0 Also note that *both* servers "pause" for ~5s between receiving the request and responding. I think that this is caused by the backend server trying to stream its body out, and the frontend server not consuming it - possibly a seperate issue. Why the custom header comes through in 2.2.4 and not in 2.0.5 I haven't looked into - but the fact that GET and HEAD give you different headers would seem to be a bug with each version yes? Anyway, applying my patch to 2.2.4: $ curl --get --verbose http://localhost:2204/rproxy/redirect * About to connect() to localhost port 2204 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2204 > GET /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2204 > Accept: */* > < HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:30:59 GMT < Server: Apache/2.2.4 (Unix) < X-My-Secret-Header: moo < Location: http://httpd.apache.org/ < Content-Length: 208 < Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://httpd.apache.org/">here</a>.</p> </body></html> * Connection #0 to host localhost left intact * Closing connection #0 So it now passes the header through correctly. Also, the HEAD is now the same: $ curl --head --verbose http://localhost:2204/rproxy/redirect * About to connect() to localhost port 2204 * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 2204 > HEAD /rproxy/redirect HTTP/1.1 > User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > Host: localhost:2204 > Accept: */* > < HTTP/1.1 302 Found HTTP/1.1 302 Found < Date: Thu, 15 Feb 2007 18:31:37 GMT Date: Thu, 15 Feb 2007 18:31:37 GMT < Server: Apache/2.2.4 (Unix) Server: Apache/2.2.4 (Unix) < X-My-Secret-Header: moo X-My-Secret-Header: moo < Location: http://httpd.apache.org/ Location: http://httpd.apache.org/ < Content-Type: text/html; charset=iso-8859-1 Content-Type: text/html; charset=iso-8859-1 * Connection #0 to host localhost left intact * Closing connection #0 And the "pause" has gone - hence my theory. I think the above all indicates: 1) 2.2.x has different behaviour from 2.0.x (as myself - and others it would seem - were expecting/relying on that, I call it a regression) in how they treat 3xx response when erroroveride is on. 2) Both branches may have a bug with HEAD not being the same as GET. 3) There's a potential issue left with bodies not being consumed (?) I'll look further into 2) and 3) tomorrow, if time allows. Have been stuck in meetings most of this afternoon, it's past home time, and I've looked at this enough already for one day. :)
(In reply to comment #11) > Yes, granted it does give a definition of "error pages" - but as you seem to > agree, most people would not take that to include redirects (or 304 responses > come to that). Sigh, see what happens when you're tired. That should obviously read: it does *not* give a definition of "error pages"
(In reply to comment #11) > 2) Both branches may have a bug with HEAD not being the same as GET. This one is slightly complex. See bug #41646 > 3) There's a potential issue left with bodies not being consumed (?) Relatively straight-forward. See bug #41644 I think these are all independent issues (well, they are related in the block of code the affect which makes for confusion testing when you start discovering them, but they can be resolved on their own).
Created attachment 19915 [details] extend directive based on Nick's comments see http://mail-archives.apache.org/mod_mbox/httpd-dev/200704.mbox/%3c20070404163031.71e4da7b@grimnir%3e
simple fix to skip override processing for 1xx and 3xx (in addition to 2xx) responses committed to trunk and proposed for backport to 2.2.x
For posterity; references to the fixes committed: trunk: http://svn.apache.org/viewvc?view=rev&rev=527969 2.2.x: http://svn.apache.org/viewvc?view=rev&rev=534068