Hi,

We are experiencing some strange problems with file-uploads through 
Pound. It seems that the time that Pound uses for it's incoming requests 
is too short for uploading files. For normal GET-requests or 
POST-requests using only plain forms, Pound works perfectly !

This is an example config for a site that needs uploads:

    Service "94.124.89.211_80__wepaintyou.massmovement.nl"
       HeadRequire "Host: wepaintyou.massmovement.nl"
       BackEnd
          Address 91.199.219.83
          Port 80
          Priority 5
          Timeout 300
       End
    End

Is it possible that "Timeout 300" is only used for the connection itself 
but individual requests have a different timeout ? Also, please note 
that uploads using Flash-sites nearly everytime succeed and it seems 
that https-uploads succeed as well...

Can someone help us ? Our client now wants to remove the loadbalancer as 
it is of "no use" to them... Can't let that happen :)

Best regards,
Marcus Smit

Hello,

MSDirect Internet Diensten - Support <support(at)msdirect.nl> (Fr 02 Dez
2011 07:49:29 CET):[...]

For my understanding an file upload is a normal POST request, using a
the multipart/form encoding.
[...]

According to the docs the Timeout is the time pound waits for the
*response* from the backend. This should start counting as soon your
request (upload) is completed.

Did you watch the connection using tcpdump? Does the data stop flowing?
Does pound kill the connection, regardless receiving data from the
client?
[...]

Hi,

Thanks for your reply and sorry for my late response. I had no way of 
repoducing this problem until yesterday. It seems that the problem only 
occurs using Windows7. I tested using Firefox.
[...]
That is how i understand it too :)
[...][...]

In the attachement you will find the tcp stream for the POST request. As 
i see it, this is just a plan POST request that is reset with a FIN/ACK. 
But why the RST's ? The site went to production so i could not test it 
as it was but instead i had to create an extra alias as you can see in 
the dump (marcustest.massmovement.nl). Points to the same virtualhost in 
Apache.

I hope someone can help me solving this problem.
------------------------------------------------------------------------
Met vriendelijke groet,
Marcus Smit
MSDirect Internet Diensten
www.msdirect.nl <http://www.msdirect.nl>
Mobiel: 06 - 167 20 817

Hi,

>>> This is an example config for a site that needs uploads:
>>>
>>>     Service "94.124.89.211_80__wepaintyou.massmovement.nl"
>>>        HeadRequire "Host: wepaintyou.massmovement.nl"
>>>        BackEnd
>>>           Address 91.199.219.83
>>>           Port 80
>>>           Priority 5
>>>           Timeout 300
>>>        End
>>>     End
>>>
>>> Is it possible that "Timeout 300" is only used for the connection
>>> itself but individual requests have a different timeout ? Also,
>>> please note that uploads using Flash-sites nearly everytime
succeed
>>> and it seems that https-uploads succeed as well...[...][...]

I think i reduced the problem to two possible bugs in either Windows7 or 
Pound.

In a session that fails, i see a retransmission (by Windows) for a 
packet that already was ACKed. Ofcourse, this packet is not ACKed again, 
which for Windows is enough reason to close the session, making the 
fileupload fail. See the attached file for more info.

As you can see in packet 12 an ACK is sent for 8281, but the Windows 7 
workstation retransmits the packet in packet 13. Nothing happens for 10 
seconds, so Pound (or Linux, not sure) cleans up.

------------------------------------------------------------------------
Met vriendelijke groet,
Marcus Smit
MSDirect Internet Diensten
www.msdirect.nl <http://www.msdirect.nl>
Mobiel: 06 - 167 20 817

Hello,

 

The FIN/ACK is a TCP stack related issue, not pound. Do you have the most
recent patches and such for your host server?

 

I’ve seen this same behavior for a variety of hosts, not just Windows
clients. Plus, I’ve seen this on CentOS 4 and CentOS 5, and so far no relief
in the way of a TCP protocol bug fix. 

 

Do you have iptables running or any other software firewall on the pound host?
I have had iptables running on all of the boxes that host pound with this
error. I have not tried disabling iptables to see if it clears.

 
[...]

Options on linux that have resolved network issues for me in the past:
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 0 > /proc/sys/net/ipv4/tcp_ecn

If they work you will want to use sysctl.conf to make them permanent, i.e.:
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_ecn=0

Other tuning here
http://www.bloke.com/linux/kernel/network.html

YMMV!

Joe

From: Jacob Anderson [mailto:jwa(at)beyond-ordinary.com]
Sent: Friday, December 16, 2011 11:37 AM
To: pound(at)apsis.ch
Subject: RE: [Pound Mailing List] Upload problem through Pound

Hello,

The FIN/ACK is a TCP stack related issue, not pound. Do you have the most
recent patches and such for your host server?

I’ve seen this same behavior for a variety of hosts, not just Windows
clients. Plus, I’ve seen this on CentOS 4 and CentOS 5, and so far no relief
in the way of a TCP protocol bug fix.

Do you have iptables running or any other software firewall on the pound host?
I have had iptables running on all of the boxes that host pound with this
error. I have not tried disabling iptables to see if it clears.
[...]

Hi,

Thanks for the responses.

I totally agree that the session initiation and maintainance is the 
responsibility of the operating system. This Centos 6.0 system is fully 
up-to-date with patches from one month ago.

I tested stopping iptables (had some weird ftp troubles some time ago 
which solved by restarting iptables), but with no luck,  I also tested 
the settings Joe Gooch suggested. Unfortunately no luck there.

Euhm... all this suddenly, the upload succeeds.. I did test some 
settings, but all changes are reverted.. Hm, how am i going to explain 
this to the customer...
------------------------------------------------------------------------
Met vriendelijke groet,
Marcus Smit
MSDirect Internet Diensten
www.msdirect.nl <http://www.msdirect.nl>
Mobiel: 06 - 167 20 817

Op 16-12-2011 19:27, Joe Gooch schreef:[...]