I'm trying to setup a transparent caching forward proxy for my LAN. In transparent mode forward proxy users are not aware they're passing through a proxy, the networking layers silently tunnel web requests through the forward proxy. I got the forward caching proxy part working (I think) : when a LAN browser is configured to use myip:myport as proxy things work I also got the networking magic working : requests sent by LAN browsers not configured to use the proxy do end up in apache, as evidenced by apache logs The problem is when I put the two together proxying is not working anymore. A browser will do a "GET http://foo.org/bar/" and the apache logs will show "GET /bar/" -> all requests are interpreted by apache as local People on the netfilter list say "Well, if Apache sees the traffic on port 8081 then your redirect is working as it should. Netfilter does nothing with the contents of the packets, just with the addresses and ports." Googling for mod_proxy+transparent yields a ton of results, but none of them seem useful. So it seems it's an apache problem, either on the code or documentation side. At least I couldn't find any solution after searching for hours.
This is not a bug, but a configuration problem. Please discuss this on users@httpd.apache.org.
That's why I wrote it might be a "documentation bug", since despite re-reading it many times I'm no closer to the solution
I don't believe this is a config problem. My understanding is that apache does not support transparent proxying. What you say in your problem report is not true: The browser does not send "GET http://foo.org/bar/" (as it would if it was configured to use a forward proxy) but rather GET /bar/ Host: foo.org In this case, Apache will treat the request as local and, not finding the host foo.org, it will serve it from the default vhost (the first listed). This is not a bug, but it could be considered a feature request.
Ok, marking it as feature request. I'm very surprised mod_proxy does not support transparency, as a lot of forward proxy setups need it these days. Am I really the first one to ask about it ? In the meanwhile, maybe it would be interesting to write it's not supported in mod_proxy docs ?
Although mod_proxy does not support transparent proxing explicitly I think you can solve this with mod_rewrite. Please add the following rule to your proxy configuration: RewriteRule (.*) http://%{HTTP_HOST}$1 [P] Keep in mind that this can only work for http.
Well, this sort of works but : 1. only if https is not proxied (meaning I can't proxy ports where I don't know if the connection will be http or https : squid,svn,http-alt,webcache) 2. it seriously messes up the browsers that use the proxy explicitely So it's still far from ideal :(
(In reply to comment #6) > Well, this sort of works but : > 1. only if https is not proxied (meaning I can't proxy ports where I don't know > if the connection will be http or https : squid,svn,http-alt,webcache) You need to know beforehand anyway if the connection is http or https because the virtual host you are redirecting your traffic to for transparent proxying needs to know at configuration time whether it needs to speak http or https. But, if you use the rule RewriteRule (.*) https://%{HTTP_HOST}$1 [P] in a SSL enabled virtual host, the client browser will get the wrong certificate (the one from your proxy). > 2. it seriously messes up the browsers that use the proxy explicitely Use a different virtual host on a different port for your transparent proxy. This should fix it.
Ok, I guess that'll mean one interface for transparent proxiing another for normal proxiing. is there a way to share definition between the two virtualhosts ? Only the rewriterule will differ. Also if I want to to self-hosted transparent proxining I need a way to recognize the proxy packets. (if I blindly redirect all messages to external 80 to local 8081 I'll end up with a loop for the proxy requests) If there a way to control the originating port mod_proxy will use ?
(In reply to comment #8) > Ok, I guess that'll mean one interface for transparent proxiing another for > normal proxiing. Not really an interface just another port. eg. 8080 for the normal proxy and 8081 for your transparent proxy. > > is there a way to share definition between the two virtualhosts ? Only the > rewriterule will differ. I do not think, that they look very similar. The transparent proxy host only consist of the rewriterule, whereas the normal proxy has all the stuff within that is needed for a forward proxy. Nevertheless, usally I share common configuration directives between different virtual hosts by placing them in a separate file and including them via include into both virtual hosts. > > Also if I want to to self-hosted transparent proxining I need a way to recognize > the proxy packets. (if I blindly redirect all messages to external 80 to local > 8081 I'll end up with a loop for the proxy requests) If there a way to control > the originating port mod_proxy will use ? No, but I think you can fix this witin your netfilter rules. I am not sure if I remember netfilter correctly, but if you place your redirect rule in the FORWARD chain this should not touch the connections from your local box.
(In reply to comment #7) > > 1. only if https is not proxied (meaning I can't proxy ports where I don't know > > if the connection will be http or https : squid,svn,http-alt,webcache) > > You need to know beforehand anyway if the connection is http or https because > the virtual host you are redirecting your traffic to for transparent proxying > needs to know at configuration time whether it needs to speak http or https. > But, if you use the rule > > RewriteRule (.*) https://%{HTTP_HOST}$1 [P] > > in a SSL enabled virtual host, the client browser will get the wrong certificate > (the one from your proxy). > Another thought on transparent SSL proxying: Even if httpd would be capable of doing this, I think this functionality would look very similar to what netfilter does with masquerading (and masquerading is handled better on the kernel level IMHO).
(In reply to comment #9) > (In reply to comment #8) > > is there a way to share definition between the two virtualhosts ? Only the > > rewriterule will differ. > > I do not think, that they look very similar. With include I ended up with <IfModule mod_proxy.c> Listen 192.168.1.1:8081 NameVirtualHost 192.168.1.1:8081 <VirtualHost 192.168.1.1:8081> RewriteEngine on RewriteRule ^(.*) http://%{HTTP_HOST}$1 [P] Include conf.d/proxy-shared.txt </VirtualHost> Listen 192.168.1.1:8080 NameVirtualHost 192.168.1.1:8080 <VirtualHost 192.168.1.1:8080> Include conf.d/proxy-shared.txt </VirtualHost> </IfModule> and all the mod_proxy directives in proxy-shared.txt. Is this wrong ? > > Also if I want to to self-hosted transparent proxining I need a way to recognize > > the proxy packets. (if I blindly redirect all messages to external 80 to local > > 8081 I'll end up with a loop for the proxy requests) If there a way to control > > the originating port mod_proxy will use ? > > No, but I think you can fix this witin your netfilter rules. I am not sure if I > remember netfilter correctly, but if you place your redirect rule in the FORWARD > chain this should not touch the connections from your local box. In this setup apache is part of the local box, but the apps you want to redirect too. (and why would you want to do that when you have control of the box, and of the apps config? Have you tried to setup proxying for all the apps of a Linux system ? Much simpler to silently redirect everything to the local apache proxy) You need a way to distinguish http requests made by you apps and the http requests made by the proxy at the netfilter level - hence I was asking if I could force the use of a special port by the proxy). So : requests on the system to port 80 silently loop back to the local apache requests on the system to port 80 from special proxy port are not redirected
(In reply to comment #10) > Another thought on transparent SSL proxying: Even if httpd would be capable of > doing this, I think this functionality would look very similar to what netfilter > does with masquerading (and masquerading is handled better on the kernel level > IMHO). You're right, I only want the proxy for caching, and this doesn't work with SSl anyway. The problem is those ports where the traffic can be http or https, I'd like to cache http and let https pass through
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.