|
/
Zope
/
Apsis
/
Pound Mailing List
/
Archive
/
2005
/
2005-07
/
FW: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
[
http logs / James Billson <james.billson(at)cim... ]
[
/ gauze(at)dropdead.org ]
FW: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-05 15:46:04 |
[ FULL ]
|
Robert,
What do you think the next step is in troubleshooting this issue?? I could
grant you access to the application and walk you through the steps necessary
to reproduce the issue. I could also generate logs from Pound and send them
to you as well.
This issue with PDFs displaying, and the browser waiting for the client
timeout to hit is the only issue we see with the Pound product. If we
bypass the proxy, the problem goes away, so we are pretty sure there is some
type of interaction between IE, Pound and the backend IIS server. Maybe
solving this problem might help some other Pound user in the future.
Thanks again for your continued help.
Steven
[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-05 16:56:06 |
[ FULL ]
|
Update - Potential Breakthrough??!!
I was able to sniff packets from the proxy to our backend server, and from
the proxy to the client and found some interesting results.
When I ran the application without the Proxy, everything ran fine as
suspected, each packet matching what was sent and received. When I ran the
application through the Proxy and decoded each packet, I found that just
before all of the client timeouts start occurring, Pound splits 1 packet
into 2, and I believe that's where the problems start.
Below I've attached the original packet from the backend server, and then
the 2 subsequent packets that Pound creates. From my simple perspective, it
seems like the browser gets confused when a packet either doesn't measure up
to the content length properly, or when a 200 packet with data gets split, I
really have no idea though.. ;-) The next packets in the sequence are all
TCP Fin packets which close the connections after the Client Timeout value
is reached. Then the client opens up fresh connections, and finishes
loading the page. (This is where I errantly thought the problem had to do
with IE and 4 connections, I thought they were all used up, but in reality,
IE is just waiting for something, and then the timeouts hit, and everything
resets and IE is happy again)
Hopefully with this information you can glean some insight into the issue
and determine the proper course of action. Thanks for your help, and I hope
I didn't confuse you too much.
Steven
These are the last packets which Pound
Here are the packets:
Original Packet:
HTTP Section: 1126 bytes
Hyper Text: HTTP/1.0 200 OK
Hyper Text Continuation: Content-Length: 1059
Hyper Text Continuation: Content-Type: text/html
Hyper Text Continuation: {Blank Line}
Hyper Text Continuation: <HTML>
Hyper Text Continuation: <HEAD>
Hyper Text Continuation: <TITLE>
Hyper Text Continuation: </TITLE>
Hyper Text Continuation: <META NAME="GENERATOR" CONTENT="BusinessObjects
6.0 on Windows">
Hyper Text Continuation: <META HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=ISO-8859-1">
Hyper Text Continuation: </HEAD>
Hyper Text Continuation: {Blank Line}
Hyper Text Continuation: <BODY onLoad=' if (parent){.if
(typeof(parent.setActiveBar) != "undefined") ..parent.setActiveBar(
Hyper Text Continuation: true); .if (typeof(parent.setNewToken) !=
"undefined") ..parent.setNewToken("wi00000001"); parent.s
Hyper Text Continuation: tatus="";} ' BGCOLOR="#003399" TEXT="#FFFFFF"
LINK="#FFCCFF" VLINK="#FFFFFF" ALINK="#0000FF">
Hyper Text Continuation: <TABLE CELLPADDING="4" CELLSPACING="0"
BORDER="0">
Hyper Text Continuation: <TR>
Hyper Text Continuation: <TD NOWRAP>
Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00001&TS=-1531696140
Hyper Text Continuation: &sPath=/Document/cover+page.pdf"
TARGET="ReportArea"><B>
Hyper Text Continuation: <FONT COLOR="#FFFFFF">
Hyper Text Continuation: Cover Page</FONT>
Hyper Text Continuation: </B>
Hyper Text Continuation: </A>
Hyper Text Continuation: </TD>
Hyper Text Continuation: <TD NOWRAP>
Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00001&TS=-1531696140
Hyper Text Continuation:
&sPath=/Document/agreement+index+by+agreement+area+.pdf"
TARGET="ReportArea"><B>
Hyper Text Continuation: <FONT COLOR="#FFFFFF">
Hyper Text Continuation: Agreement Index by Agreement Area </FONT>
Hyper Text Continuation: </B>
Hyper Text Continuation: </A>
Hyper Text Continuation: </TD>
Hyper Text Continuation: </TR>
Hyper Text Continuation: </TABLE>
Hyper Text Continuation: </BODY>
Hyper Text Continuation: </HTML>
---------------------------------------------------------------
Pound splits the packet above into 2 separate packets, but doesn't correct
the content length.
HTTP Section: 67 bytes
Hyper Text: HTTP/1.0 200 OK
Hyper Text Continuation: Content-Length: 1059
Hyper Text Continuation: Content-Type: text/html
Hyper Text Continuation: {Blank Line}
---------------------------------------------------------------
HTTP Section: 1059 bytes
Hyper Text: <HTML>
Hyper Text Continuation: <HEAD>
Hyper Text Continuation: <TITLE>
Hyper Text Continuation: </TITLE>
Hyper Text Continuation: <META NAME="GENERATOR" CONTENT="BusinessObjects
6.0 on Windows">
Hyper Text Continuation: <META HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=ISO-8859-1">
Hyper Text Continuation: </HEAD>
Hyper Text Continuation: {Blank Line}
Hyper Text Continuation: <BODY onLoad=' if (parent){.if
(typeof(parent.setActiveBar) != "undefined") ..parent.setActiveBar(
Hyper Text Continuation: true); .if (typeof(parent.setNewToken) !=
"undefined") ..parent.setNewToken("wi00000004"); parent.s
Hyper Text Continuation: tatus="";} ' BGCOLOR="#003399" TEXT="#FFFFFF"
LINK="#FFCCFF" VLINK="#FFFFFF" ALINK="#0000FF">
Hyper Text Continuation: <TABLE CELLPADDING="4" CELLSPACING="0"
BORDER="0">
Hyper Text Continuation: <TR>
Hyper Text Continuation: <TD NOWRAP>
Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00004&TS=-1531965343
Hyper Text Continuation: &sPath=/Document/cover+page.pdf"
TARGET="ReportArea"><B>
Hyper Text Continuation: <FONT COLOR="#FFFFFF">
Hyper Text Continuation: Cover Page</FONT>
Hyper Text Continuation: </B>
Hyper Text Continuation: </A>
Hyper Text Continuation: </TD>
Hyper Text Continuation: <TD NOWRAP>
Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00004&TS=-1531965343
Hyper Text Continuation:
&sPath=/Document/agreement+index+by+agreement+area+.pdf"
TARGET="ReportArea"><B>
Hyper Text Continuation: <FONT COLOR="#FFFFFF">
Hyper Text Continuation: Agreement Index by Agreement Area </FONT>
Hyper Text Continuation: </B>
Hyper Text Continuation: </A>
Hyper Text Continuation: </TD>
Hyper Text Continuation: </TR>
Hyper Text Continuation: </TABLE>
Hyper Text Continuation: </BODY>
Hyper Text Continuation: </HTML>
---------------------------------------------------------------
[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-05 17:04:25 |
[ FULL ]
|
On Tue, 5 Jul 2005 08:46:04 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
could[...]
necessary[...]
send them[...]
client[...]
is some[...]
Maybe[...]
Thanks for the offer, but I'm afraid it doesn't work that way (too many
liability issues). If your employer is in a position to get paid support
that would be fine (mail info(at)apsis.ch for details), but my people would
get a fit if I were to just access somebody's servers like this.
As to what you CAN do: as a test please stick two instances of tcpwatch,
one between the client and Pound and one between Pound and the back-end
server. Let's look at the logs produced and we'll take it from there.[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-05 17:18:40 |
[ FULL ]
|
On Tue, 5 Jul 2005 09:56:06 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
from[...]
ran the[...]
just[...]
packet[...]
then[...]
perspective, it[...]
measure up[...]
split, I[...]
are all[...]
value[...]
do[...]
reality,[...]
everything[...]
issue[...]
I hope[...]
Sorry, but this isn't it. HTTP works over TCP sockets (aka SOCK_STREAM)
and thus is never even aware of the TCP packets. It does not matter how
many packets there are - they are automatically reassembled into a data
stream before the application ever sees them. The only thing that
matters is if the data contents (split over however many packets and/or
fragments) is the same.
BTW, the Content-length header indicates the HTTP payload size - it has
nothing to do with the TCP payload.
Try sending a larger file (anything over 1.5K - that is the normal TCP
payload limit) and you'll see the HTTP response split over multiple TCP
packets.
All the above is a description of how TCP/IP is defined. It may well be
the case that Microsoft have implemented it differently in Windows -
let's test that. Start with larger responses to see how your client
behaves over multiple packets.[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-06 17:16:59 |
[ FULL ]
|
Hi Robert,
I've done more investigation at both the HTTP level and TCP level. The
packets at the HTTP level are virtually identical whether or not I use the
proxy (shouldn't be a big surprise). However, when I sniff the packets on
the wire, I find that not every TCP packet sent by Pound is ACK'd by the
client. Correct me if I'm wrong, but every TCP packet needs to be ACK'd,
regardless if Pound split the packets or not, right? I see, at least on my
sample, that 4 packets went unACK'd, then there was a delay of 20 seconds
(the Client timeout value), then Pound tore down the connections to the
server and client, and then the client rebuilt the connections and finished
loading the page.
Could the missing ACKs be causing the problem I am seeing? I am using a
Windows package called Observer to sniff the packets. I am running Pound on
Linux and can create a TCPDUMP file for you, but I'm not sure of the
switches for TCPDUMP to get the information you need.
The missing ACKs are pretty repeatable, if I refresh the page, the missing
ACKs occur in the virtually the same places over and over again. One place
it's repeatable is when Pound split the HTTP request across multiple
packets, in my example below where Pound split the 200 message from the rest
of the content, could be coincidence.
I hope I'm not barking up the wrong tree, and I'm trying to do the best
troubleshooting I can given my limited knowledge of HTTP and TCP/IP. I
appreciate your patience and education in getting me up to speed to
troubleshoot such a nagging problem.
Thanks again for your help,
Steven
[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-06 23:29:19 |
[ FULL ]
|
Update,
If I go into IE's settings and tell it not to use HTTP1.1 at all, the
application works fine without any delay. It problem seems to happen only
when HTTP1.1 is selected at the browser level.
Hope that helps.
Steven
[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-07 13:27:31 |
[ FULL ]
|
On Wed, 6 Jul 2005 10:16:59 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
If the ACKs are really missing then you have major networking issues
(nothing to do with Pound or IE - this happens at the TCP stack level).
I suggest you check carefully, as some implementations support sliding
windows, where several ACKs are sent at once.[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-07 13:29:07 |
[ FULL ]
|
On Wed, 6 Jul 2005 16:29:19 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
Did you try my suggestion about the two tcpwatch instances? That would
give you the best information about what happens at the HTTP level
(rather than TCP, which can be confusing).[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-07 20:20:57 |
[ FULL ]
|
Hi Robert,
I setup 2 instances of tcpwatch, one between the browser and Pound, and the
other between Pound and IIS, and configured the browser and Pound to go
through the tcpwatch's. The request/responses were identical (with the
exception of the X-Forwarded header from Pound).
Attached are the request/responses from the client to Pound when a refresh
is done on the page in question. What doesn't show up in this log are the
"timings" of the request/responses. All of the transactions occur
sub-second, except for the Packet 3 response, where the server closes the
connection between the browser and Pound after 20 seconds, not in the
sub-second range like the other transactions, then the Packet 4 request
triggers. All of the connections between Pound and IIS close sub-second as
well.
I'm not sure what I'm looking for in each of the request/responses, tcpwatch
doesn't seem to indicate any sort of error. I hope this information makes
sense to you.
Steven
[...]
|
| Attachments: | | |
| tcpwatch.zip |
application/x-zip-compressed |
7662 Bytes |
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-08 15:22:01 |
[ FULL ]
|
On Thu, 7 Jul 2005 13:20:57 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
Did you edit the logs in any way? There are some very bizarre things in
there, such as a different number of GETs and responses, responses that
start (or do not end) in the expected way (see leading spaces in an 200
OK, responses that do not end in a CRLF), etc.
Was there any difference between the logs between the client and Pound
and the logs between Pound and the back-end?[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-09 02:51:09 |
[ FULL ]
|
Hi Robert,
[...]
I did not edit the files at all in any way. The messages are exactly as
tcpwatch generated them. Looking at the asp code which generates these
responses, it doesn't surprise me that they look weird. If there are
specific things which should be modified to make them better behaved, please
let me know, and I will change what I can. The software is a commercial
application, but it doesn't surprise me that the code isn't "clean".
[...]
I used a program called Examdiff Pro to compare the files between the
browser and pound, and pound and IIS, and the only lines it picked up on
were the added X-Forward headers that Pound adds, otherwise, Examdiff found
the files to be identical, spaces, special characters, CRLF and all.
[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-15 15:36:55 |
[ FULL ]
|
I looked again at your transcripts, and googled some. I see that you use
"Cache-control: private" in your responses. Please try replacing that
with "Cache-control: no-cache" and possibly "Pragma: no-cache" (i am
sure you'll find some IIS parameter for that). Some PHP applications
have shown that to be beneficial when talking to IE.
Let me know if this helps.[...]
|
|
|
RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
"Steven Vlach" <Home_Boy(at)msn.com> |
2005-07-18 23:08:28 |
[ FULL ]
|
[...]
I changed the Cache-control to no-cache with no change in results. I
couldn't force IIS to use the Pragma: no-cache, maybe because it's a HTTP
1.0 command, and IIS is using HTTP 1.1??
Is there a way to force Pound to use HTTP 1.0 to the backend server for HTTP
like there is for HTTPS and IE?? Something like a NoHTTP 1 setting?? If I
force the browser to use HTTP 1.0, the problem goes away, so I figure if I
could tell Pound to use HTTP 1.0 to the backend, that would fix the issue
and eliminate me from configuring every browser to use 1.0. Just a thought.
Thanks for continuing to look at this problem.
Steven
[...]
|
|
|
Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE and saving attachments
Robert Segall <roseg(at)apsis.ch> |
2005-07-19 14:52:46 |
[ FULL ]
|
On Mon, 18 Jul 2005 16:08:28 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:
[...]
The Pragma is indeed 1.0
[...]
No, there is no way, and I very much doubt it would help.[...]
|
|
|
|
