Bug 37355 - Allow to specify Proxy-Authorization in ProxyRemote
Summary: Allow to specify Proxy-Authorization in ProxyRemote
Status: REOPENED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy (show other bugs)
Version: 2.4.33
Hardware: Other Linux
: P2 enhancement with 16 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords: MassUpdate, PatchAvailable
Depends on:
Blocks: 34837 40808
  Show dependency tree
 
Reported: 2005-11-04 11:52 UTC by Hendrik Harms
Modified: 2019-07-05 12:16 UTC (History)
4 users (show)



Attachments
ProxyRemote extension (14.19 KB, patch)
2006-01-19 08:15 UTC, Hendrik Harms
Details | Diff
Patched ported to 2.2.x (8.87 KB, application/octet-stream)
2011-11-22 22:40 UTC, Francois Leurent
Details
A first (dirty but stable) stab at a port to 2.4.x. (13.19 KB, patch)
2012-05-22 12:28 UTC, intersat2
Details | Diff
patch for modules/proxy of httpd-2.4.6 (19.29 KB, patch)
2013-12-16 21:40 UTC, Hendrik Harms
Details | Diff
patch for modules/proxy of httpd-2.4.10 (17.55 KB, text/plain)
2015-01-10 16:47 UTC, Hendrik Harms
Details
mod_proxy 2.4.12 error_log (1.48 KB, text/plain)
2015-04-21 08:56 UTC, Michael Göhler
Details
patch for modules/proxy of httpd-2.4.16 (19.17 KB, text/plain)
2015-07-31 10:42 UTC, Hendrik Harms
Details
patch for modules/proxy of httpd-2.4.23 (19.27 KB, text/plain)
2016-09-19 07:10 UTC, Hendrik Harms
Details
patch for modules/proxy of httpd-2.4.33 (22.32 KB, patch)
2018-04-26 08:06 UTC, Chris
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hendrik Harms 2005-11-04 11:52:08 UTC
The apache reverse proxy tries to establish a https connecteion to the http
proxy. This causes an error:
  [info] SSL Proxy connect failed
  [info] SSL Library Error: 336031996 error:140770FC:SSL
routines:SSL23_GET_SERVER_HELLO:unknown protocol

Configuration:
  LoadModule proxy_module modules/mod_proxy.so
  LoadModule proxy_connect_module modules/mod_proxy_connect.so
  LoadModule proxy_http_module modules/mod_proxy_http.so
  LoadModule ssl_module modules/mod_ssl.so

  SSLProxyEngine On
  ProxyRequests Off
  ProxyRemote https://foo.org http://proxy.bar.org:8080
  <Location /foo
    ProxyPass https://foo.org
    ProxyPassReverse https://foo.org
  </Location>
Comment 1 Hendrik Harms 2005-11-15 15:22:52 UTC
The problem is the missing CONNECT Handshake. If a Proxy is defined
(ProxyRemote) and the scheme to backend is HTTPS then a CONNECT has to be send
to the proxy to open the proxy tunnel.
Comment 2 Hendrik Harms 2005-11-25 10:37:43 UTC
If I may take the patch from bug 19188, I am still missing a way to add a
Proxy-Authorization Header.

nice to have:
   ProxyRemote https://foo.org http://[<login>:<password>@]proxy.bar.org:8080

The "<login>:<password>@" should be an optional feature of the ProxyRemote
config command.
Comment 3 Hendrik Harms 2006-01-19 08:15:09 UTC
Created attachment 17448 [details]
ProxyRemote extension

This patch based on apache-httpd version 2.0.55. It adds a new feature for the
ProxyRemote config item. It is now possible to set an optional 
<proxy-user:proxy-passwd@> prefix. This works with http and https backends.
  ProxyRemote <protocol://backend-server[:port]>
<protocol://[<proxy-user>:<proxy-passwd>@]<proxy-server[:port]>
Example:
  ProxyRemote https://foo.org http://user:passwd@bar.com:8080

I could only test this patch with a BlueCoat and a Squid Proxy.
Comment 4 Francois Leurent 2011-11-22 22:40:33 UTC
Created attachment 27977 [details]
Patched ported to 2.2.x

This is the same thing, 5 years later.
This patch based on apache-httpd version 2.2.21. It adds a new feature for the
ProxyRemote config item. It is now possible to set an optional 
<proxy-user:proxy-passwd@> prefix. This works with http and https backends.
  ProxyRemote <protocol://backend-server[:port]>
<protocol://[<proxy-user>:<proxy-passwd>@]<proxy-server[:port]>
Example:
  ProxyRemote https://foo.org http://user:passwd@bar.com:8080

I could only test this patch with a Squid Proxy.
Comment 5 YosheE 2012-04-16 08:47:59 UTC
Hello François,

I have rebuilt apache 2.2 with your patch, it works well for http request so useful, thank you. But my remote proxy sends me a 407 error (like if authentication was not provided) for the https requests. Here is my config for https vhost :

NameVirtualHost *:443
<VirtualHost *:443>
ServerName my.server.name
SSLEngine on
SSLProxyEngine on
SSLVerifyClient none
SSLCertificateFile "/my/cert/file.crt"
SSLCertificateKeyFile "/my/key/file.key"
ProxyRemote https https://user:pass@proxyhost:80
ProxyPass / https://foreign.https.server/
ProxyPassReverse / https://foreign.https.server/
</VirtualHost>

I can see in apache logs :
[error] send_http_connect: the forward proxy returned code is '407'

and apache sends to the end browser a 503 error code.

Is there something wrong in my config or do you have an idea regarding the patch ?

Thank you very much,
Julien

(In reply to comment #4)
> Created attachment 27977 [details]
> Patched ported to 2.2.x
> 
> This is the same thing, 5 years later.
> This patch based on apache-httpd version 2.2.21. It adds a new feature for the
> ProxyRemote config item. It is now possible to set an optional 
> <proxy-user:proxy-passwd@> prefix. This works with http and https backends.
>   ProxyRemote <protocol://backend-server[:port]>
> <protocol://[<proxy-user>:<proxy-passwd>@]<proxy-server[:port]>
> Example:
>   ProxyRemote https://foo.org http://user:passwd@bar.com:8080
> 
> I could only test this patch with a Squid Proxy.
Comment 6 intersat2 2012-05-22 12:28:03 UTC
Created attachment 28815 [details]
A first (dirty but stable) stab at a port to 2.4.x.
Comment 7 Hendrik Harms 2013-12-16 21:40:39 UTC
Created attachment 31121 [details]
patch for modules/proxy of httpd-2.4.6

Yes it's me - 8 years later and still the same problem ;-)
I took the last patch (thanks to intersat2, YosheE and Francois Leurent) and added some stuff to pass a HTTP Forward Proxy with HTTPS. By doing so I ran on an other problem (Bug 55892) also fixed by this patch.

I only could test this patch on Linux passing through a Squid and a BlueCoat forward proxy
Comment 8 Hendrik Harms 2015-01-10 16:47:17 UTC
Created attachment 32361 [details]
patch for modules/proxy of httpd-2.4.10

Patch adapted to httpd-2.4.10
This patch includes also the patch for Bug 55892 and the patch for Bug 57138
Comment 9 Hendrik Harms 2015-01-10 16:52:19 UTC
there is a typo in my last comment: wrong: Bug 57138 -
correct: the patch of Bug 57139 was included into my last patch 32361.
Comment 10 victor_amrich 2015-03-19 23:55:25 UTC
Hello:

I have the same problem in a server in which we are using a squid, the version of the apache is 2.2.14. My Operating System is SLES 11 SP3, can i apply the patch uploaded by Francois Leurent, or there is another?

Many Thanks!
Victor
Comment 11 Michael Göhler 2015-04-21 08:56:33 UTC
Created attachment 32670 [details]
mod_proxy 2.4.12 error_log

Hi Hendrik,

I applied your patch for 2.4.10. and still observe a strange behavior in our scenario:

client -> mod_proxy 2.4.12 -> squid -> IIS 8.5

Here's my config:

LogLevel debug
<VirtualHost *:80>
ServerName my.server.name
SSLProxyEngine on
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
ProxyRemote https http://proxyhost:8080
ProxyPass / https://foreign.https.server/
ProxyPassReverse / https://foreign.https.server/
</VirtualHost>

Every request after 120 sec idle returns a HTTP 502. Subsequent connections work as expected, if there is no delay of >120 sec.

error_log content attachment.

Suggestions very appreciated.

Many Thanks,
Michael
Comment 12 Hendrik Harms 2015-04-22 08:55:08 UTC
(In reply to Michael Göhler from comment #11)
> 
> client -> mod_proxy 2.4.12 -> squid -> IIS 8.5
> 
> ProxyRemote https http://proxyhost:8080
>

You don't need this patch for your setup - no user/password is configured in ProxyRemote. Only the patches of Bug 57139 and may be of Bug 55892 are needed for your setup.

> 
> Every request after 120 sec idle returns a HTTP 502. Subsequent connections
> work as expected, if there is no delay of >120 sec.
> 

The 502 usually indicates a problem during establishing the TCP connection. What does the squid do with idle connections? Do you have a firewall in your setup or other routing devices dropping their routing informations without TCP-close after an idle timeout?
Comment 13 Michael Göhler 2015-04-24 15:03:01 UTC
(In reply to Hendrik Harms from comment #12)

Thanks for your reply!

> You don't need this patch for your setup - no user/password is configured in
> ProxyRemote. Only the patches of Bug 57139 and may be of Bug 55892 are
> needed for your setup.

I know. It was just the easiest way to combine both.

> The 502 usually indicates a problem during establishing the TCP connection.
> What does the squid do with idle connections? Do you have a firewall in your
> setup or other routing devices dropping their routing informations without
> TCP-close after an idle timeout?

I have a 120 sec. timeout on squid which matches what we see.

The Apache correctly identifies the connection as stale and reestablishes it.
AH00951: HTTPS: backend socket is disconnected.
AH00949: send_http_connect: response from the forward proxy: HTTP/1.0 200 Connection established

It seams to be an SSL handshake issue, as

the second request logs:
[ssl:info] AH01964: Connection to child 0 established (server my.server.name:80)
[ssl:debug] ssl_engine_kernel.c(1378): AH02275: Certificate Verification ...

while the first logs:
[ssl:info] AH01964: Connection to child 0 established (server my.server.name:80)
[ssl:info] AH02003: SSL Proxy connect failed

I'll go for some LogLevel=trace3 logs, as the error seams to happen in SSL_connect().

Just a side-note:
Are you aware of the discussion in Bug 55892? They decided to go without the force-proxy-request-1.0 condition, which is still part of your patch.
Comment 14 Michael Göhler 2015-04-28 11:33:44 UTC
(In reply to Michael Göhler from comment #13)

Solved my issue using
SetEnv proxy-nokeepalive 1
SetEnv proxy-initial-not-pooled 1
Comment 15 William A. Rowe Jr. 2015-07-28 21:46:28 UTC
Hello again Hendrik.

Glad we were able to get PR 55892 addressed now in 2.4 (and 2.2).

AIUI the patch above (2015-01-10) addressed both https: CONNECT remoteing as well as auth.  I would expect some redundancy/collision there?

Looking at the changes, I'm uncertain of whether the group would entertain any API changes to mod_proxy.h for feature enhancements such as changing the args list for auth. Third party module authors need binary stability between subversion releases, and that would include those writing any mod_proxy framework participants.

If there was a way to store/pass this info without altering the API, it's much more likely to be applied to 2.4.  I was prepared to refactor your patch for trunk/2.4.16 changes to eliminate duplication of PR 55892 concerns, and commit to trunk, but I'm looking for your thoughts before proceeding.  Thank you for the submissions and for fighting the good fight of getting these changes into httpd!
Comment 16 Hendrik Harms 2015-07-31 10:02:43 UTC
(In reply to William A. Rowe Jr. from comment #15)

> Glad we were able to get PR 55892 addressed now in 2.4 (and 2.2).
Thank you for this - I have already found it in 2.4.16 :-)
 
> AIUI the patch above (2015-01-10) addressed both https: CONNECT remoteing as
> well as auth.  I would expect some redundancy/collision there?
I don't see a redundancy or collision between CONNECT remote an proxy authentication.
I've build this patch because I need a setup discribed in Bug 55892:
The backend of my apache reverse proxy is place behind a http forward proxy. This forward proxy requires a proxy authentication (HTTP-407) and the backend requires https. The clients sending requests to my reverse proxy should not know anything about this setup especially the needed proxy authentication. This should be covered by the reverse proxy.
As described somethere in the RFCs it should be possible to configure the proxy authentication as a hop-to-hop header. So the ReverseProxy should be able to authenticate itself to the forward proxy.
A forward proxy could be defined by the ProxyRemote config directive. So I thought it was a good idea to enhance this config directive by an optional authentication prefix [<user>:<password>@]

> Looking at the changes, I'm uncertain of whether the group would entertain
> any API changes to mod_proxy.h for feature enhancements such as changing the
> args list for auth. Third party module authors need binary stability between
> subversion releases, and that would include those writing any mod_proxy
> framework participants.
Yes, unfortunately my patch will kill the compatibility of some third party modules :-(
But I think best place to store and transfer this authentication info should be very close to the other attributes defining the forward proxy (hostname, port, ... + auth), because they belong together. 
 
> If there was a way to store/pass this info without altering the API, it's
> much more likely to be applied to 2.4. 
If the API change is not possible in 2.4 we should flag it for 2.6 and think about a workaround in 2.4

> I was prepared to refactor your
> patch for trunk/2.4.16 changes to eliminate duplication of PR 55892
> concerns, and commit to trunk, but I'm looking for your thoughts before
> proceeding.  Thank you for the submissions and for fighting the good fight
> of getting these changes into httpd!
HTH - never give up ;-)
Comment 17 Hendrik Harms 2015-07-31 10:42:57 UTC
Created attachment 32948 [details]
patch for modules/proxy of httpd-2.4.16

Patches for Bug 55892 and Bug Bug 57139 are applied to 2.4.16 - so they are not part of this patch anymore.
Comment 18 William A. Rowe Jr. 2015-08-06 20:31:28 UTC
Thank you for the revised patch.  It is easier for the core developers to read a small dedicated patch that addresses just a single issue at a time, even if that means there is a set of several patches to get to the desired outcome.

Here's one proposal that wouldn't require API changes.

If we added a 407 handler extension of some sort, we would be able to handle specific backend challenges for auth credentials in a unified way.  If we stored the credentials in an auth challenge table, we could eventually support other auth methods, including digest or even cookie based auth.

The syntax of the ProxyRemote directive still changes, but the scheme://host would be split from the auth user:pass and stored in two different mechanisms.

The only downside is whether users in practice would want to identify a particular proxy use by use case.  While we can keep different stores of auth tupples by virtual host, further differentiation might prove troublesome.

Thoughts?
Comment 19 owensy83 2015-09-17 09:37:54 UTC
Apologies for the ignorance here, after googling I came across this patch and it is exactly what I need....but how do I get this change into my mod_proxy.so file??
Comment 20 Hendrik Harms 2016-09-19 07:10:05 UTC
Created attachment 34271 [details]
patch for modules/proxy of httpd-2.4.23
Comment 21 Hendrik Harms 2016-09-19 07:12:48 UTC
Pardon - it's still the same kind of patch like those before
Comment 22 Mathieu CARBONNEAUX 2016-12-28 18:26:23 UTC
adding proxy credential in proxyremote make possible to chaine proxy with differente authentication from front to back in chaining proxy and make possible to use authenticated proxy on backend proxy...

when they are integrated in last http 2.4 version?
Comment 23 Chris 2018-04-26 08:06:49 UTC
Created attachment 35893 [details]
patch for modules/proxy of httpd-2.4.33

Hi there,
based on the previous patches in this thread I created a new version which works for httpd-2.4.33.
Have a nice day,
 Chris
Comment 24 William A. Rowe Jr. 2018-11-07 21:09:28 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.
Comment 25 Christophe JAILLET 2018-11-09 22:08:07 UTC
Looks like this one is still valid on 2.4.x branch.
Re-opening and setting version to what is said in comment 23
Comment 26 Hendrik Harms 2018-12-03 10:29:43 UTC
this enhancement is still open for 2.4.33 and later
Comment 27 basinilya@gmail.com 2019-05-02 05:44:55 UTC
This is all wrong. The ProxyRemote directive should not exist. Forwarding proxy requests upstream could be handled by ProxyPass. 

Actually, ProxyRemote has the ability to forward upstream all unknown http methods and "CONNECT" could be among them.

The proxy module just needs to not strip the schema and server from the request url.

The authentication could then be configured with ProxyAddHeaders.
Comment 28 basinilya@gmail.com 2019-05-02 06:25:38 UTC
(In reply to basinilya@gmail.com from comment #27)
> The authentication could then be configured with ProxyAddHeaders.

Sorry, I meant "RequestHeader add Proxy-Authorization"

By the way, it works for GET, but not for CONNECT, even with "early". Is this a bug?
Comment 29 Sebastian 2019-07-05 11:46:33 UTC
(In reply to basinilya@gmail.com from comment #28)
> (In reply to basinilya@gmail.com from comment #27)
> > The authentication could then be configured with ProxyAddHeaders.
> 
> Sorry, I meant "RequestHeader add Proxy-Authorization"
> 
> By the way, it works for GET, but not for CONNECT, even with "early". Is
> this a bug?

Same here - ProxyRemote with RequestHeader set Proxy-Authorization works fine until using CONNECT.
While GET sends the RequestHeader, CONNECT doesn't.

Did you already found out a working solution for this?
Comment 30 basinilya@gmail.com 2019-07-05 12:16:40 UTC
(In reply to Sebastian from comment #29)
> Same here - ProxyRemote with RequestHeader set Proxy-Authorization works
> fine until using CONNECT.
> While GET sends the RequestHeader, CONNECT doesn't.
> 
> Did you already found out a working solution for this?

The workaround is to use Haproxy https://pastebin.com/dhuACwJg unless you're on Windows. I tried nginx, apache and even ISS, but all of them lack this feature.