"Dave Steinberg" <dave(at)redterror.net> wrote in 
message news:MailBoxer.1238.1252935504.52.pound(at)apsis.ch...
> Saumil Shah wrote:
>> Greetings,
>>
>> The expected format for all XFF headers is that there's only ONE XFF
>> header. Every proxy server appends the client IP to the end of the
>> XFF IP list.
>
> Not so!  RFC 2616, section 4.2 says:
>
> Multiple message-header fields with the same field-name MAY be present in 
> a message if and only if the entire field-value for that header field is 
> defined as a comma-separated list [i.e., #(values)]. It MUST be possible 
> to combine the multiple header fields into one "field-name: field-value" 
> pair, without changing the semantics of the message, by appending each 
> subsequent field-value to the first, each separated by a comma. The order 
> in which header fields with the same field-name are received is therefore 
> significant to the interpretation of the combined field value, and thus a 
> proxy MUST NOT change the order of these field values when a message is 
> forwarded.
>
> So pound's implementation is in accordance with the standard.  Most 
> webservers do the munge-to-one-field operation before log parsers or 
> applications see it, so I don't think there's any issue.

Sorry for reopening an old thread, but I was just looking into this and 
noticed the same thing that the original poster found; that instead of 
appending the source's IP to the XFF header, it creates a new one.

While Dave's interpretation of the standard may seem correct, there is a 
significant problem with Pound's implementation.  Given that there is no 
official XFF standard, we need to look at the de facto standard for XFF. 
All other proxy's and LBs seem to append the last source IP to the XFF in a 
comma-delimited string.

By Pound not doing the same thing, and instead creating a new XFF header, I 
see 2 problems.
1) There is no way to know the exact sequence of the headers.
2) Any downstream proxy / LB may not append the source IP to the correct XFF 
header.

Consequently, given two XFF headers, there is no way to accurately 
reconstruct the sequence of proxy/lb IPs.

For example:
XFF: 198.20.123.123, 10.1.1.1, 192.168.1.17, 169.123.1.1

In this instance, we know that the client IP was 198.20.123.123 and can see 
the order of the LB/Proxys: (ie: first was 10.1.1.1, then 192.168.1.7, then 
169.123.1.1).

If we look at an example using pound, you might get
XFF: 198.20.123.123, 192.168.1.17
XFF: 10.1.1.1
XFF: 169.123.1.1

Can you still accurately reconstruct the sequence of Proxy/LB's?


I would strongly vote for updating Pound's use of the XFF to be more 
compliant with the defacto standard that most other LB/Proxy's seem to 
follow.

Is there an area where one can post bug requests / fixes / etc?

Thanks,

Eric

At 16:08 22/10/2009, Eric B. wrote:
>1) There is no way to know the exact sequence of the headers.

As already pointed out by Dave, the HTTP RFC explicitly says that
header: a
header: b

is exactly equivalent to:
header: a,b

So the sequence is perfectly defined.

Anyway, I don't think anybody ever cares about anything but the last value :-)

Jacques.

"Jacques Caron" <jc(at)oxado.com> wrote in 
message news:MailBoxer.1297.1256226086.27.pound(at)apsis.ch...
> At 16:08 22/10/2009, Eric B. wrote:
>>1) There is no way to know the exact sequence of the headers.
>
> As already pointed out by Dave, the HTTP RFC explicitly says that
> header: a
> header: b
>
> is exactly equivalent to:
> header: a,b
>
> So the sequence is perfectly defined.

Yes, but can we be sure that subsequent proxies/lbs will always append to 
the final header in the list?  I don't have Squid or anything else 
installed, so am not sure of it, but am going on assumption that they 
probably just grab the first XFF that they find.

Any confirmation (or rejection) of this behaviour would be appreciated.

Thanks,

Eric

El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:
> At 16:08 22/10/2009, Eric B. wrote:
> >1) There is no way to know the exact sequence of the headers.
> 
> As already pointed out by Dave, the HTTP RFC explicitly says that
> header: a
> header: b
> 
> is exactly equivalent to:
> header: a,b
> 
> So the sequence is perfectly defined.

The problem is that X-Forwarded-For definitios is exactly the opposite.



-- 
Iñaki Baz Castillo <ibc(at)aliax.net>

El Jueves, 22 de Octubre de 2009, Eric B. escribió:
> "Jacques Caron" <jc(at)oxado.com> wrote in
> message news:MailBoxer.1297.1256226086.27.pound(at)apsis.ch...
> 
> > At 16:08 22/10/2009, Eric B. wrote:
> >>1) There is no way to know the exact sequence of the headers.
> >
> > As already pointed out by Dave, the HTTP RFC explicitly says that
> > header: a
> > header: b
> >
> > is exactly equivalent to:
> > header: a,b
> >
> > So the sequence is perfectly defined.
> 
> Yes, but can we be sure that subsequent proxies/lbs will always append to
> the final header in the list?  I don't have Squid or anything else
> installed, so am not sure of it, but am going on assumption that they
> probably just grab the first XFF that they find.
> 
> Any confirmation (or rejection) of this behaviour would be appreciated.

Since there is no a real standard for this, most probably most of the proxy do 
a unexpected behavior when adding X-Forwarded-For having the requests already 
such a header.

-- 
Iñaki Baz Castillo <ibc(at)aliax.net>

At 17:07 22/10/2009, Eric B. wrote:
>Yes, but can we be sure that subsequent proxies/lbs will always append to
>the final header in the list?  I don't have Squid or anything else
>installed, so am not sure of it, but am going on assumption that they
>probably just grab the first XFF that they find.

If they do that I would say they're not compliant, and I actually 
believe they don't do that but actually append XFF at the end. A 
quick check through the squid source tells me it actually merges 
multiple headers with the same name into a CSL and adds XFF at the 
end. I expect everyone to do either that or use the pound approach, 
and add a new header at the end.

BTW, even though XFF is not defined in a standard, the Via header is, 
and it is assembled in the same order as XFF, so it's all very consistent.

Jacques.

Hi,

At 17:11 22/10/2009, Iñaki Baz Castillo wrote:
>El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:
> > At 16:08 22/10/2009, Eric B. wrote:
> > >1) There is no way to know the exact sequence of the headers.
> >
> > As already pointed out by Dave, the HTTP RFC explicitly says that
> > header: a
> > header: b
> >
> > is exactly equivalent to:
> > header: a,b
> >
> > So the sequence is perfectly defined.
>
>The problem is that X-Forwarded-For definitios is exactly the opposite.

No. Whatever happens, the equivalence is valid. Then there is the 
question of what order should you read that in (i.e. did the request 
go through a first and then b, or the opposite), which is de facto 
defined as "a then b", probably for consistency with the Via header 
which works the same.

Jacques.

> "Iñaki Baz Castillo" <ibc(at)aliax.net> wrote 
> in message news:MailBoxer.1300.1256228434.18.pound(at)apsis.ch...
> El Jueves, 22 de Octubre de 2009, Eric B. escribió:
> > "Jacques Caron" <jc(at)oxado.com> wrote in
> > message 
> > news:MailBoxer.1297.1256226086.27.pound(at)apsis.ch...
> >
> > > At 16:08 22/10/2009, Eric B. wrote:
> > >>1) There is no way to know the exact sequence of the headers.
> > >
> > > As already pointed out by Dave, the HTTP RFC explicitly says that
> > > header: a
> > > header: b
> > >
> > > is exactly equivalent to:
> > > header: a,b
> > >
> > > So the sequence is perfectly defined.
> >
> > Yes, but can we be sure that subsequent proxies/lbs will always append 
> > to
> > the final header in the list?  I don't have Squid or anything else
> > installed, so am not sure of it, but am going on assumption that they
> > probably just grab the first XFF that they find.
> >
> > Any confirmation (or rejection) of this behaviour would be appreciated.
>
> Since there is no a real standard for this, most probably most of the 
> proxy do
> a unexpected behavior when adding X-Forwarded-For having the requests 
> already
> such a header.

I agree.  Which is why I would vote for Pound to keep in line with the other 
major players out there, like Squid, and append the source IP to the XFF 
header as opposed to adding a new header every time.

Eric 

Iñaki Baz Castillo wrote:
> El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:
>> At 16:08 22/10/2009, Eric B. wrote:
>>> 1) There is no way to know the exact sequence of the headers.
>> As already pointed out by Dave, the HTTP RFC explicitly says that
>> header: a
>> header: b
>>
>> is exactly equivalent to:
>> header: a,b
>>
>> So the sequence is perfectly defined.
> 
> The problem is that X-Forwarded-For definitios is exactly the opposite.

I still don't see how pound is doing things wrong here.  If it gets a 
request like:

X-Forwarded-For: client1, proxy1

and then the header sent to the backend is:

X-Forwarded-For: client1, proxy1
X-Forwarded-For: proxy2

then that is correct based on the reconstruction rules cited previously 
and the definition here:

http://en.wikipedia.org/wiki/X-Forwarded-For

Is pound doing something other than what I've described above?

Regards,
-- 
Dave Steinberg
http://www.geekisp.com/
http://www.steinbergcomputing.com/

>> Since there is no a real standard for this, most probably most of the 
>> proxy do
>> a unexpected behavior when adding X-Forwarded-For having the requests 
>> already
>> such a header.
> 
> I agree.  Which is why I would vote for Pound to keep in line with the other 
> major players out there, like Squid, and append the source IP to the XFF 
> header as opposed to adding a new header every time.

Is there actually a problem?  Because in my setup things *seem* to be 
working fine.

Regards,
-- 
Dave Steinberg
http://www.geekisp.com/
http://www.steinbergcomputing.com/

> "Dave Steinberg" <dave(at)redterror.net> wrote in 
> message news:MailBoxer.1303.1256230199.98.pound(at)apsis.ch...
> Iñaki Baz Castillo wrote:
> > El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:
> >> At 16:08 22/10/2009, Eric B. wrote:
> >>> 1) There is no way to know the exact sequence of the headers.
> >> As already pointed out by Dave, the HTTP RFC explicitly says that
> >> header: a
> >> header: b
> >>
> >> is exactly equivalent to:
> >> header: a,b
> >>
> >> So the sequence is perfectly defined.
> >
> > The problem is that X-Forwarded-For definitios is exactly the opposite.
>
> I still don't see how pound is doing things wrong here.  If it gets a 
> request like:
>
> X-Forwarded-For: client1, proxy1
>
> and then the header sent to the backend is:
>
> X-Forwarded-For: client1, proxy1
> X-Forwarded-For: proxy2
>
> then that is correct based on the reconstruction rules cited previously 
> and the definition here:
>
> http://en.wikipedia.org/wiki/X-Forwarded-For
>
> Is pound doing something other than what I've described above?

No - pound is doing that.  My question is what happens if this request then 
goes thorugh Squid (or another LB/Proxy).  Does the LB/Proxy expect a single 
XFF and simply append it to the first one it finds, or does it specifically 
look for the last XFF and append to that one.

ie: will the result of a proxy3 be:
XFF: client1, proxy1
XFF: proxy2 (pound), proxy3

or

XFF: client1, proxy1, proxy3
XFF: proxy2

If the former, then no problem.  If the later, then there is a bigger issue 
to be dealt with....

For me, I have noticed this as I am using Tomcat behind pound to serve my 
Java app, and in order to avoid any IP problems, am using a Tomcat Valve to 
reconstruct the original protocol/source ports for the request so that my 
app doesn't actually need to change.  In the Valve, I noticed that they are 
not accounting for multiple XFF headers; not a big deal - I can fix that. 
However, it then begged the bigger question of whether or not downstream 
proxies will work properly with Pound's extra header, when Pound seems to be 
the only Proxy out there that adds it in...

Thanks,

Eric

El Jueves, 22 de Octubre de 2009, Jacques Caron escribió:
> >The problem is that X-Forwarded-For definitios is exactly the opposite.
> 
> No. Whatever happens, the equivalence is valid. Then there is the 
> question of what order should you read that in (i.e. did the request 
> go through a first and then b, or the opposite), which is de facto 
> defined as "a then b",

> probably for consistency with the Via header which works the same.

It's curious, in SIP protocol (more or less based on HTTP) all the headers 
(including Via header) work in the opposite way, this is:

- A client sends a SIP request to proxy1.org.
- proxy1.org relays the request to proxy2.org.
- proxy2.org relays the request to the destination/server.

The request arriving to destination/server could look like (all the options 
mean the same):

a)
Via: SIP/2.0/TCP proxy2.org, SIP/2.0/UDP proxy1.org
 
b)
Via: SIP/2.0/TCP proxy2.org
Via: SIP/2.0/UDP proxy1.org


Anyhow I've checked that RFC2616 states that Via in HTTP should use the 
opposite order.



-- 
Iñaki Baz Castillo <ibc(at)aliax.net>