/ Zope / Apsis / Pound Mailing List / Archive / 2009 / 2009-10 / Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request

[ << ] [ >> ]

[ proxy only for wget..... / Jean-Pierre van Melis ... ] [ Re: Pound is converting error 413 (size limit ... ]

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
"Eric B." <ebenze(at)hotmail.com>
2009-10-22 17:08:58 [ FULL ]
"Dave Steinberg" <dave(at)redterror.net> wrote in 
message news:MailBoxer.1238.1252935504.52.pound(at)apsis.ch...[...][...][...]

Sorry for reopening an old thread, but I was just looking into this and 
noticed the same thing that the original poster found; that instead of 
appending the source's IP to the XFF header, it creates a new one.

While Dave's interpretation of the standard may seem correct, there is a 
significant problem with Pound's implementation.  Given that there is no 
official XFF standard, we need to look at the de facto standard for XFF. 
All other proxy's and LBs seem to append the last source IP to the XFF in a 
comma-delimited string.

By Pound not doing the same thing, and instead creating a new XFF header, I 
see 2 problems.
1) There is no way to know the exact sequence of the headers.
2) Any downstream proxy / LB may not append the source IP to the correct XFF 
header.

Consequently, given two XFF headers, there is no way to accurately 
reconstruct the sequence of proxy/lb IPs.

For example:
XFF: 198.20.123.123, 10.1.1.1, 192.168.1.17, 169.123.1.1

In this instance, we know that the client IP was 198.20.123.123 and can see 
the order of the LB/Proxys: (ie: first was 10.1.1.1, then 192.168.1.7, then 
169.123.1.1).

If we look at an example using pound, you might get
XFF: 198.20.123.123, 192.168.1.17
XFF: 10.1.1.1
XFF: 169.123.1.1

Can you still accurately reconstruct the sequence of Proxy/LB's?


I would strongly vote for updating Pound's use of the XFF to be more 
compliant with the defacto standard that most other LB/Proxy's seem to 
follow.

Is there an area where one can post bug requests / fixes / etc?

Thanks,

Eric

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Jacques Caron <jc(at)oxado.com>
2009-10-22 17:40:53 [ FULL ]
At 16:08 22/10/2009, Eric B. wrote:[...]

As already pointed out by Dave, the HTTP RFC explicitly says that
header: a
header: b

is exactly equivalent to:
header: a,b

So the sequence is perfectly defined.

Anyway, I don't think anybody ever cares about anything but the last value :-)

Jacques.

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
"Eric B." <ebenze(at)hotmail.com>
2009-10-22 18:07:26 [ FULL ]
"Jacques Caron" <jc(at)oxado.com> wrote in 
message news:MailBoxer.1297.1256226086.27.pound(at)apsis.ch...[...][...][...]

Yes, but can we be sure that subsequent proxies/lbs will always append to 
the final header in the list?  I don't have Squid or anything else 
installed, so am not sure of it, but am going on assumption that they 
probably just grab the first XFF that they find.

Any confirmation (or rejection) of this behaviour would be appreciated.

Thanks,

Eric

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Iñaki Baz Castillo <ibc(at)aliax.net>
2009-10-22 18:11:09 [ FULL ]
El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:[...]

The problem is that X-Forwarded-For definitios is exactly the opposite.


[...]

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
=?utf-8?q?I=C3=B1aki_Baz_Castillo?= <ibc(at)aliax.net>
2009-10-22 18:20:22 [ FULL ]
El Jueves, 22 de Octubre de 2009, Eric B. escribió:[...]

Since there is no a real standard for this, most probably most of the proxy do 
a unexpected behavior when adding X-Forwarded-For having the requests already 
such a header.
[...]

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Jacques Caron <jc(at)oxado.com>
2009-10-22 18:36:21 [ FULL ]
At 17:07 22/10/2009, Eric B. wrote:[...]

If they do that I would say they're not compliant, and I actually 
believe they don't do that but actually append XFF at the end. A 
quick check through the squid source tells me it actually merges 
multiple headers with the same name into a CSL and adds XFF at the 
end. I expect everyone to do either that or use the pound approach, 
and add a new header at the end.

BTW, even though XFF is not defined in a standard, the Via header is, 
and it is assembled in the same order as XFF, so it's all very consistent.

Jacques.

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Jacques Caron <jc(at)oxado.com>
2009-10-22 18:38:11 [ FULL ]
Hi,

At 17:11 22/10/2009, Iñaki Baz Castillo wrote:[...]

No. Whatever happens, the equivalence is valid. Then there is the 
question of what order should you read that in (i.e. did the request 
go through a first and then b, or the opposite), which is de facto 
defined as "a then b", probably for consistency with the Via header 
which works the same.

Jacques.

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
"Eric B." <ebenze(at)hotmail.com>
2009-10-22 18:42:56 [ FULL ]
> "Iñaki Baz Castillo" <ibc(at)aliax.net> wrote [...]

I agree.  Which is why I would vote for Pound to keep in line with the other 
major players out there, like Squid, and append the source IP to the XFF 
header as opposed to adding a new header every time.

Eric

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Dave Steinberg <dave(at)redterror.net>
2009-10-22 18:44:43 [ FULL ]
Iñaki Baz Castillo wrote:[...][...]
>>> 1) There is no way to know the exact sequence of the
headers.[...][...]

I still don't see how pound is doing things wrong here.  If it gets a 
request like:

X-Forwarded-For: client1, proxy1

and then the header sent to the backend is:

X-Forwarded-For: client1, proxy1
X-Forwarded-For: proxy2

then that is correct based on the reconstruction rules cited previously 
and the definition here:

http://en.wikipedia.org/wiki/X-Forwarded-For

Is pound doing something other than what I've described above?

Regards,[...]

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Dave Steinberg <dave(at)redterror.net>
2009-10-22 19:13:44 [ FULL ]
>> Since there is no a real standard for this, most probably most of the
[...][...]

Is there actually a problem?  Because in my setup things *seem* to be 
working fine.

Regards,[...]

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
"Eric B." <ebenze(at)hotmail.com>
2009-10-22 20:21:43 [ FULL ]
> "Dave Steinberg" <dave(at)redterror.net> wrote in [...]

No - pound is doing that.  My question is what happens if this request then 
goes thorugh Squid (or another LB/Proxy).  Does the LB/Proxy expect a single 
XFF and simply append it to the first one it finds, or does it specifically 
look for the last XFF and append to that one.

ie: will the result of a proxy3 be:
XFF: client1, proxy1
XFF: proxy2 (pound), proxy3

or

XFF: client1, proxy1, proxy3
XFF: proxy2

If the former, then no problem.  If the later, then there is a bigger issue 
to be dealt with....

For me, I have noticed this as I am using Tomcat behind pound to serve my 
Java app, and in order to avoid any IP problems, am using a Tomcat Valve to 
reconstruct the original protocol/source ports for the request so that my 
app doesn't actually need to change.  In the Valve, I noticed that they are 
not accounting for multiple XFF headers; not a big deal - I can fix that. 
However, it then begged the bigger question of whether or not downstream 
proxies will work properly with Pound's extra header, when Pound seems to be 
the only Proxy out there that adds it in...

Thanks,

Eric

Re: [Pound Mailing List] Pound: "X-Forwarded-For" should be deleted if present in the request
Iñaki Baz Castillo <ibc(at)aliax.net>
2009-10-23 11:30:57 [ FULL ]
El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:[...]
[...]

It's curious, in SIP protocol (more or less based on HTTP) all the headers 
(including Via header) work in the opposite way, this is:

- A client sends a SIP request to proxy1.org.
- proxy1.org relays the request to proxy2.org.
- proxy2.org relays the request to the destination/server.

The request arriving to destination/server could look like (all the options 
mean the same):

a)
Via: SIP/2.0/TCP proxy2.org, SIP/2.0/UDP proxy1.org
 
b)
Via: SIP/2.0/TCP proxy2.org
Via: SIP/2.0/UDP proxy1.org


Anyhow I've checked that RFC2616 states that Via in HTTP should use the 
opposite order.


[...]

MailBoxer