Bug 55635

Hello Ivan,

Thanks for reporting this.

I have been trying to replicate your setup, but I am getting different results.
I am using httpd trunk on a Linux system.
Perhaps you are running a different version of httpd?

I've made a few changes that should still be equivalent.

1)  I changed the LogFormat as follows to make the log entries a little easier for me to read:

LogFormat "%h %a %{c}a xf=\"%{X-Forwarded-For}i\" %l %u %t \"%m\" \"%r&\" \"%q&\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" pid=%{pid}P tid=%{tid}P time_ms=%D" combined

2)  I am using curl instead of Apache HTTP Client similar to the following:

curl -v -H "X-Forwarded-For: 1.1.1.2, 1.1.1.1, 87.245.198.54, 87.250.250.203" http://srv2-x64rh6-01:1280/test/1.xml 

3)  I changed the value of RemoteIPInternalProxy to match my own client server. (which is also a private network IP like yours)

4)  The rest of the addresses are exactly the same as yours.

The result I am getting is equivalent to the following changes to your results:

access_log.2013.10.07:
ivoronin.net.billing.ru 1.1.1.2 172.20.106.70 xf="-" - - [07/Oct/2013:12:44:00 +0400] "GET" "GET /test/1.xml HTTP/1.1&" "&" 403 212 "-" "Apache-HttpClient/4.1.2 (java 1.5)" pid=27844 tid=140346537215744 time_ms=3111

In other words, the client IP is changed to the first IP address in the X-Forwarded-For list and the X-Forwarded-For header is cleared.

If I change the Allow to "Allow from localhost 127.0.0.1 1.1.1.2", I get the following equivalent result:

access_log.2013.10.07:
ivoronin.net.billing.ru 1.1.1.2 172.20.106.70 xf="1.1.1.2" - - [07/Oct/2013:12:44:00 +0400] "GET" "GET /test/1.xml HTTP/1.1&" "&" 403 212 "-" "Apache-HttpClient/4.1.2 (java 1.5)" pid=27844 tid=140346537215744 time_ms=3111

Take care,

Mike Rumph

Hello Mike,

You got the similar results.
The X-Forwarded-For must be "1.1.1.2, 1.1.1.1, 87.245.198.54", because "87.245.198.54" is not trusted.
But you got "-" in first case and "1.1.1.2" in another.

I think that it's a bug because it's not possible to pass real Client IP to server behind this Apache server.

Best regards,
Ivan Voronin.

Hello Ivan,

I have been studying the mod_remoteip documentation and code to try to make sense of the results that we are seeing.
I think that I finally understand the results that you are getting.
And it appears to me at this point that your results are as they should be according to the documentation at the following link:
- http://httpd.apache.org/docs/trunk/mod/mod_remoteip.html 

I will try to explain this, but it is not easy.
Here is how I see it now:

mod_remoteip processes the contents of X-Forwarded-For from right to left in cycles of a while loop after your RemoteIPInternalProxy and RemoteIPTrustedProxy proxies are added to a proxy match list.

Cycle 1:
The code begins with X-Forwarded-For equal to "1.1.1.2, 1.1.1.1, 87.245.198.54, 87.250.250.203" and the client IP is equal to "172.20.106.70".
The client IP is compared against the proxy match list.
172.20.106.70 is listed as an internal proxy.
So its view of the X-Forwarded-For list is trusted.
So 87.250.250.203 is interpreted as a valid useragent IP address.
So 87.250.250.203 becomes the client IP and is removed from the X-Forwarded-For list.

Cycle 2:
X-Forwarded-For is equal to "1.1.1.2, 1.1.1.1, 87.245.198.54" and the client IP is equal to "87.250.250.203".
87.250.250.203 is listed as a trusted proxy.
So its view of the X-Forwarded-For list is trusted.
So 87.245.198.54 is interpreted as a valid useragent IP address.
So 87.245.198.54 becomes the client IP and is removed from the X-Forwarded-For list.

Cycle 3:
X-Forwarded-For is equal to "1.1.1.2, 1.1.1.1" and the client IP is equal to "87.245.198.54".
87.245.198.54 is not an internal or trusted proxy.
So the cycles stop.

Final mod_remoteip result":
X-Forwarded-For is equal to "1.1.1.2, 1.1.1.1" and the client IP is equal to "87.245.198.54".

And this is the result that you are seeing.

I think the key point here is that RemoteIPInternalProxy and RemoteIPTrustedProxy refer to trusted proxies not trusted clients.
As a trusted proxy it can be relied upon to have added a trusted client IP to the end of the X-Forwarded-For list before forwarding the request on to the backend server.

Now I think that what should happen next is for mod_proxy to take over.
If mod_proxy is properly configured and 87.245.198.54 is accepted as an allowed client IP, then mod_proxy should add 87.245.198.54 to the end of X-Forwarded-For.
This is what is not happening.

It could be because you do not have 87.245.198.54 included in the "Allow from" directive.
You could also take a look at the ProxyAddHeaders directive.
- http://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxyaddheaders
What proxy directives are you using in your configuration?

This bug is related to bug 55637.
And I will try to update that bug report with an explanation as well.

I hope that this analysis is helpful.

Take care,

Mike Rumph

You have made a claim in comment #1 that...

access_log.2013.10.07:
ivoronin.net.billing.ru 87.245.198.54 172.20.106.70 1.1.1.2, 1.1.1.1 - - [07/Oct/2013:12:44:00 +0400] "GET" "GET /test/1.xml HTTP/1.1&" "&" 403 212 "-" "Apache-HttpClient/4.1.2 (java 1.5)" pid=27844 tid=140346537215744 time_ms=3111

is inaccurate.  It is the correct result.  The processed request is presented 
by the IP 87.245.198.54, which claims it is forwarding for 1.1.1.2, 1.1.1.1
while the internal proxy 172.20.106.70 and trusted extranet proxy 87.250.250.203
are both 'erased' from the x-forwarded-for and apparent IP address of the req
(except as presented by the connection's IP address).

The 1.1.1.2 and 1.1.1.1 IP addresses are not presented as the user-agent's IP
address *because* the 87.245.198.54 host is *not* trusted.

An IP address would never simultaniously appear to be the user agent's IP addr
while also appearing in the X-Forwarded-For list.  Such combinations make no sense.

This behavior is by design.