Robert,

What do you think the next step is in troubleshooting this issue??  I could
grant you access to the application and walk you through the steps necessary
to reproduce the issue.  I could also generate logs from Pound and send them
to you as well.

This issue with PDFs displaying, and the browser waiting for the client
timeout to hit is the only issue we see with the Pound product.  If we
bypass the proxy, the problem goes away, so we are pretty sure there is some
type of interaction between IE, Pound and the backend IIS server.  Maybe
solving this problem might help some other Pound user in the future.  

Thanks again for your continued help.

Steven



> -----Original Message-----
> From: Steven Vlach [mailto:Home_Boy(at)msn.com]
> Sent: Tuesday, June 28, 2005 10:18 AM
> To: 'pound(at)apsis.ch'
> Subject: RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> Hi Robert,
> 
> > -----Original Message-----
> > From: Robert Segall [mailto:roseg(at)apsis.ch]
> > Sent: Tuesday, June 28, 2005 9:59 AM
> > To: pound(at)apsis.ch
> > Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> > and saving attachments
> >
> > On Tue, 28 Jun 2005 09:08:38 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I can see how the scenario below works, and agree with how the
> > timeouts
> > > operate in Pound and the reasoning behind them.
> > >
> > > There are actually 2 "problems" we are experiencing, and if one gets
> > fixed,
> > > then the other issue is moot.  The problem you describe below is the
> > one
> > > issue we are having with a low client timeout issue affecting the
> > ability of
> > > the user to respond to an "attachment" disposition ie. saving a file
> > quick
> > > enough for Pound.  Fair enough, I can increase the timeout value and
> > give
> > > the user enough time to choose a file location etc.
> > >
> > > The other issue we are having is that another part of our web
> > application
> > > (Business Objects) serves up PDF files to be displayed in the browser,
> > and
> > > somehow the pdf does not display in the IE browser *until the client
> > timeout
> > > value is reached.  I am not sure, but I believe this issue has
> > something to
> > > do with IE opening up exactly 4 connections, and Pound needing to
> > "expire"
> > > those connections in order for the last html to arrive at the browser.
> >  In
> > > this application (maybe all), IE doesn't display the pdf until all of
> > the
> > > html for the page is delivered, which causes the delay, maybe due to
> > > buffering or something.  The application serves up the pdf quickly,
> > but
> > > additional labels on the screen don't arrive until after the client
> > timeout
> > > value occurs, which delays the pdf from displaying. The web server
> > serves up
> > > http 1.1 responses for everything except the pdf generation, then it
> > > generates http 1.0 responses (which is generated by a server-side
> > dll).
> > > This change in protocols may be confusing the browser, I'm not sure.
> >
> > Unlikely. BTW - it seems to be the other way around - 1.0 for
> > everything, 1.1 for the last. This casts some doubt on the "4
> > descriptors" idea, as 1.0 connections are closed immediately on
> > completion.
> >
> > > So, you see, if I raise the client timeout issue to say, 30 seconds,
> > then
> > > each pdf generation is delayed for 30 seconds, but the user has 30
> > seconds
> > > to respond to the save attachment dialog.  If I decrease the client
> > timeout
> > > to 1 (which I read is required for certain IE applications with
> > Pound), then
> > > pdfs generate quickly, but the user has only 1 second to choose a save
> > > location, not practical.
> > >
> > > I'm including the headers for the pdf generation to see if you can
> > shed
> > > light on this issue. I apologize for the size of the headers and
> > > information.  There is a 10 second delay between packet #4 and packet
> > #5,
> > > which coincides with a client timeout value of 10.  If I make the
> > client
> > > timeout = 30, then the delay between packet 4 and 5 becomes 30
> > seconds.
> >
> > I think I would try to lower the Server timeout. If your server is fast
> > enough this should cause no problems, and once the server-side
> > connection times out Pound will close it, as well as the client-side.
> > Why don't you try a "Server 5" and see if it helps?
> 
> 
> I tried changing the server timeout to 5, but the application isn't fast
> enough sometimes, so I made Server 10 and set the Client 20, but the PDF
> didn't generate until the client timeout (20 seconds) hit, it didn't seem
> to make a difference what the serve timeout was.
> 
> 
> >
> > > I want to thank you and the others who have taken interest in this
> > issue.
> > > We have used Pound for over 2 years now, and really appreciate the
> > quality
> > > of the code, the flexibility of the configurations, and the features
> > which
> > > are included which aren't available in other products.
> > >
> > > Steven
> >
> > Thanks for the kind words. We always appreciate getting useful user
> > comments, especially in such hard-to-diagnose cases.
> > --
> > Robert Segall
> > Apsis GmbH
> > Postfach, Uetikon am See, CH-8707
> > Tel: +41-44-920 4904
> >
> > --
> > To unsubscribe send an email with subject 'unsubscribe' to
> pound(at)apsis.ch.
> > Please contact roseg(at)apsis.ch for questions.
> > http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> > 06/1118697088000/1119970712000

Update - Potential Breakthrough??!!

I was able to sniff packets from the proxy to our backend server, and from
the proxy to the client and found some interesting results.

When I ran the application without the Proxy, everything ran fine as
suspected, each packet matching what was sent and received.  When I ran the
application through the Proxy and decoded each packet, I found that just
before all of the client timeouts start occurring, Pound splits 1 packet
into 2, and I believe that's where the problems start.

Below I've attached the original packet from the backend server, and then
the 2 subsequent packets that Pound creates.  From my simple perspective, it
seems like the browser gets confused when a packet either doesn't measure up
to the content length properly, or when a 200 packet with data gets split, I
really have no idea though..  ;-)  The next packets in the sequence are all
TCP Fin packets which close the connections after the Client Timeout value
is reached.  Then the client opens up fresh connections, and finishes
loading the page. (This is where I errantly thought the problem had to do
with IE and 4 connections, I thought they were all used up, but in reality,
IE is just waiting for something, and then the timeouts hit, and everything
resets and IE is happy again)

Hopefully with this information you can glean some insight into the issue
and determine the proper course of action.  Thanks for your help, and I hope
I didn't confuse you too much.

Steven



These are the last packets which Pound 
Here are the packets:

Original Packet:

HTTP Section: 1126 bytes
    Hyper Text: HTTP/1.0 200 OK
    Hyper Text Continuation: Content-Length: 1059 
    Hyper Text Continuation: Content-Type: text/html
    Hyper Text Continuation: {Blank Line}
    Hyper Text Continuation: <HTML>
    Hyper Text Continuation: <HEAD>
    Hyper Text Continuation: <TITLE>
    Hyper Text Continuation: &nbsp;</TITLE>
    Hyper Text Continuation: <META NAME="GENERATOR" CONTENT="BusinessObjects
6.0 on Windows">
    Hyper Text Continuation: <META HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=ISO-8859-1">
    Hyper Text Continuation: </HEAD>
    Hyper Text Continuation: {Blank Line}
    Hyper Text Continuation: <BODY  onLoad=' if (parent){.if
(typeof(parent.setActiveBar) != "undefined") ..parent.setActiveBar(
    Hyper Text Continuation: true); .if (typeof(parent.setNewToken) !=
"undefined") ..parent.setNewToken("wi00000001"); parent.s
    Hyper Text Continuation: tatus="";} ' BGCOLOR="#003399" TEXT="#FFFFFF"
LINK="#FFCCFF" VLINK="#FFFFFF" ALINK="#0000FF">
    Hyper Text Continuation: <TABLE CELLPADDING="4" CELLSPACING="0"
BORDER="0">
    Hyper Text Continuation: <TR>
    Hyper Text Continuation: <TD NOWRAP>
    Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00001&TS=-1531696140
    Hyper Text Continuation: &sPath=/Document/cover+page.pdf"
TARGET="ReportArea"><B>
    Hyper Text Continuation: <FONT COLOR="#FFFFFF">
    Hyper Text Continuation: Cover Page</FONT>
    Hyper Text Continuation: </B>
    Hyper Text Continuation: </A>
    Hyper Text Continuation: </TD>
    Hyper Text Continuation: <TD NOWRAP>
    Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00001&TS=-1531696140
    Hyper Text Continuation:
&sPath=/Document/agreement+index+by+agreement+area+.pdf"
TARGET="ReportArea"><B>
    Hyper Text Continuation: <FONT COLOR="#FFFFFF">
    Hyper Text Continuation: Agreement Index by Agreement Area </FONT>
    Hyper Text Continuation: </B>
    Hyper Text Continuation: </A>
    Hyper Text Continuation: </TD>
    Hyper Text Continuation: </TR>
    Hyper Text Continuation: </TABLE>
    Hyper Text Continuation: </BODY>
    Hyper Text Continuation: </HTML>
   ---------------------------------------------------------------    


Pound splits the packet above into 2 separate packets, but doesn't correct
the content length.

  
HTTP Section: 67 bytes
    Hyper Text: HTTP/1.0 200 OK
    Hyper Text Continuation: Content-Length: 1059 
    Hyper Text Continuation: Content-Type: text/html
    Hyper Text Continuation: {Blank Line}
   ---------------------------------------------------------------    


HTTP Section: 1059 bytes
    Hyper Text: <HTML>
    Hyper Text Continuation: <HEAD>
    Hyper Text Continuation: <TITLE>
    Hyper Text Continuation: &nbsp;</TITLE>
    Hyper Text Continuation: <META NAME="GENERATOR" CONTENT="BusinessObjects
6.0 on Windows">
    Hyper Text Continuation: <META HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=ISO-8859-1">
    Hyper Text Continuation: </HEAD>
    Hyper Text Continuation: {Blank Line}
    Hyper Text Continuation: <BODY  onLoad=' if (parent){.if
(typeof(parent.setActiveBar) != "undefined") ..parent.setActiveBar(
    Hyper Text Continuation: true); .if (typeof(parent.setNewToken) !=
"undefined") ..parent.setNewToken("wi00000004"); parent.s
    Hyper Text Continuation: tatus="";} ' BGCOLOR="#003399" TEXT="#FFFFFF"
LINK="#FFCCFF" VLINK="#FFFFFF" ALINK="#0000FF">
    Hyper Text Continuation: <TABLE CELLPADDING="4" CELLSPACING="0"
BORDER="0">
    Hyper Text Continuation: <TR>
    Hyper Text Continuation: <TD NOWRAP>
    Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00004&TS=-1531965343
    Hyper Text Continuation: &sPath=/Document/cover+page.pdf"
TARGET="ReportArea"><B>
    Hyper Text Continuation: <FONT COLOR="#FFFFFF">
    Hyper Text Continuation: Cover Page</FONT>
    Hyper Text Continuation: </B>
    Hyper Text Continuation: </A>
    Hyper Text Continuation: </TD>
    Hyper Text Continuation: <TD NOWRAP>
    Hyper Text Continuation: <A
HREF="/wiasp/bin/iswi.dll/cookie/wiqt/queryTechnique/ExtractPDF?sEntry=wi000
00004&TS=-1531965343
    Hyper Text Continuation:
&sPath=/Document/agreement+index+by+agreement+area+.pdf"
TARGET="ReportArea"><B>
    Hyper Text Continuation: <FONT COLOR="#FFFFFF">
    Hyper Text Continuation: Agreement Index by Agreement Area </FONT>
    Hyper Text Continuation: </B>
    Hyper Text Continuation: </A>
    Hyper Text Continuation: </TD>
    Hyper Text Continuation: </TR>
    Hyper Text Continuation: </TABLE>
    Hyper Text Continuation: </BODY>
    Hyper Text Continuation: </HTML>
   ---------------------------------------------------------------    





> -----Original Message-----
> From: Steven Vlach [mailto:Home_Boy(at)msn.com]
> Sent: Tuesday, July 05, 2005 8:46 AM
> To: pound(at)apsis.ch
> Subject: FW: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> Robert,
> 
> What do you think the next step is in troubleshooting this issue??  I
> could
> grant you access to the application and walk you through the steps
> necessary
> to reproduce the issue.  I could also generate logs from Pound and send
> them
> to you as well.
> 
> This issue with PDFs displaying, and the browser waiting for the client
> timeout to hit is the only issue we see with the Pound product.  If we
> bypass the proxy, the problem goes away, so we are pretty sure there is
> some
> type of interaction between IE, Pound and the backend IIS server.  Maybe
> solving this problem might help some other Pound user in the future.
> 
> Thanks again for your continued help.
> 
> Steven
> 
> 
> 
> > -----Original Message-----
> > From: Steven Vlach [mailto:Home_Boy(at)msn.com]
> > Sent: Tuesday, June 28, 2005 10:18 AM
> > To: 'pound(at)apsis.ch'
> > Subject: RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> > and saving attachments
> >
> > Hi Robert,
> >
> > > -----Original Message-----
> > > From: Robert Segall [mailto:roseg(at)apsis.ch]
> > > Sent: Tuesday, June 28, 2005 9:59 AM
> > > To: pound(at)apsis.ch
> > > Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts,
> IE
> > > and saving attachments
> > >
> > > On Tue, 28 Jun 2005 09:08:38 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > I can see how the scenario below works, and agree with how the
> > > timeouts
> > > > operate in Pound and the reasoning behind them.
> > > >
> > > > There are actually 2 "problems" we are experiencing, and if one gets
> > > fixed,
> > > > then the other issue is moot.  The problem you describe below is the
> > > one
> > > > issue we are having with a low client timeout issue affecting the
> > > ability of
> > > > the user to respond to an "attachment" disposition ie. saving a file
> > > quick
> > > > enough for Pound.  Fair enough, I can increase the timeout value and
> > > give
> > > > the user enough time to choose a file location etc.
> > > >
> > > > The other issue we are having is that another part of our web
> > > application
> > > > (Business Objects) serves up PDF files to be displayed in the
> browser,
> > > and
> > > > somehow the pdf does not display in the IE browser *until the client
> > > timeout
> > > > value is reached.  I am not sure, but I believe this issue has
> > > something to
> > > > do with IE opening up exactly 4 connections, and Pound needing to
> > > "expire"
> > > > those connections in order for the last html to arrive at the
> browser.
> > >  In
> > > > this application (maybe all), IE doesn't display the pdf until all
> of
> > > the
> > > > html for the page is delivered, which causes the delay, maybe due to
> > > > buffering or something.  The application serves up the pdf quickly,
> > > but
> > > > additional labels on the screen don't arrive until after the client
> > > timeout
> > > > value occurs, which delays the pdf from displaying. The web server
> > > serves up
> > > > http 1.1 responses for everything except the pdf generation, then it
> > > > generates http 1.0 responses (which is generated by a server-side
> > > dll).
> > > > This change in protocols may be confusing the browser, I'm not sure.
> > >
> > > Unlikely. BTW - it seems to be the other way around - 1.0 for
> > > everything, 1.1 for the last. This casts some doubt on the "4
> > > descriptors" idea, as 1.0 connections are closed immediately on
> > > completion.
> > >
> > > > So, you see, if I raise the client timeout issue to say, 30 seconds,
> > > then
> > > > each pdf generation is delayed for 30 seconds, but the user has 30
> > > seconds
> > > > to respond to the save attachment dialog.  If I decrease the client
> > > timeout
> > > > to 1 (which I read is required for certain IE applications with
> > > Pound), then
> > > > pdfs generate quickly, but the user has only 1 second to choose a
> save
> > > > location, not practical.
> > > >
> > > > I'm including the headers for the pdf generation to see if you can
> > > shed
> > > > light on this issue. I apologize for the size of the headers and
> > > > information.  There is a 10 second delay between packet #4 and
> packet
> > > #5,
> > > > which coincides with a client timeout value of 10.  If I make the
> > > client
> > > > timeout = 30, then the delay between packet 4 and 5 becomes 30
> > > seconds.
> > >
> > > I think I would try to lower the Server timeout. If your server is
> fast
> > > enough this should cause no problems, and once the server-side
> > > connection times out Pound will close it, as well as the client-side.
> > > Why don't you try a "Server 5" and see if it helps?
> >
> >
> > I tried changing the server timeout to 5, but the application isn't fast
> > enough sometimes, so I made Server 10 and set the Client 20, but the PDF
> > didn't generate until the client timeout (20 seconds) hit, it didn't
> seem
> > to make a difference what the serve timeout was.
> >
> >
> > >
> > > > I want to thank you and the others who have taken interest in this
> > > issue.
> > > > We have used Pound for over 2 years now, and really appreciate the
> > > quality
> > > > of the code, the flexibility of the configurations, and the features
> > > which
> > > > are included which aren't available in other products.
> > > >
> > > > Steven
> > >
> > > Thanks for the kind words. We always appreciate getting useful user
> > > comments, especially in such hard-to-diagnose cases.
> > > --
> > > Robert Segall
> > > Apsis GmbH
> > > Postfach, Uetikon am See, CH-8707
> > > Tel: +41-44-920 4904
> > >
> > > --
> > > To unsubscribe send an email with subject 'unsubscribe' to
> > pound(at)apsis.ch.
> > > Please contact roseg(at)apsis.ch for questions.
> > > http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> > > 06/1118697088000/1119970712000
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000

On Tue, 5 Jul 2005 08:46:04 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> Robert,
> 
> What do you think the next step is in troubleshooting this issue??  I
could
> grant you access to the application and walk you through the steps
necessary
> to reproduce the issue.  I could also generate logs from Pound and
send them
> to you as well.
> 
> This issue with PDFs displaying, and the browser waiting for the
client
> timeout to hit is the only issue we see with the Pound product.  If we
> bypass the proxy, the problem goes away, so we are pretty sure there
is some
> type of interaction between IE, Pound and the backend IIS server. 
Maybe
> solving this problem might help some other Pound user in the future.  
> 
> Thanks again for your continued help.
> 
> Steven

Thanks for the offer, but I'm afraid it doesn't work that way (too many
liability issues). If your employer is in a position to get paid support
that would be fine (mail info(at)apsis.ch for details), but my people would
get a fit if I were to just access somebody's servers like this.

As to what you CAN do: as a test please stick two instances of tcpwatch,
one between the client and Pound and one between Pound and the back-end
server. Let's look at the logs produced and we'll take it from there.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

On Tue, 5 Jul 2005 09:56:06 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> Update - Potential Breakthrough??!!
> 
> I was able to sniff packets from the proxy to our backend server, and
from
> the proxy to the client and found some interesting results.
> 
> When I ran the application without the Proxy, everything ran fine as
> suspected, each packet matching what was sent and received.  When I
ran the
> application through the Proxy and decoded each packet, I found that
just
> before all of the client timeouts start occurring, Pound splits 1
packet
> into 2, and I believe that's where the problems start.
> 
> Below I've attached the original packet from the backend server, and
then
> the 2 subsequent packets that Pound creates.  From my simple
perspective, it
> seems like the browser gets confused when a packet either doesn't
measure up
> to the content length properly, or when a 200 packet with data gets
split, I
> really have no idea though..  ;-)  The next packets in the sequence
are all
> TCP Fin packets which close the connections after the Client Timeout
value
> is reached.  Then the client opens up fresh connections, and finishes
> loading the page. (This is where I errantly thought the problem had to
do
> with IE and 4 connections, I thought they were all used up, but in
reality,
> IE is just waiting for something, and then the timeouts hit, and
everything
> resets and IE is happy again)
> 
> Hopefully with this information you can glean some insight into the
issue
> and determine the proper course of action.  Thanks for your help, and
I hope
> I didn't confuse you too much.

Sorry, but this isn't it. HTTP works over TCP sockets (aka SOCK_STREAM)
and thus is never even aware of the TCP packets. It does not matter how
many packets there are - they are automatically reassembled into a data
stream before the application ever sees them. The only thing that
matters is if the data contents (split over however many packets and/or
fragments) is the same.

BTW, the Content-length header indicates the HTTP payload size - it has
nothing to do with the TCP payload.

Try sending a larger file (anything over 1.5K - that is the normal TCP
payload limit) and you'll see the HTTP response split over multiple TCP
packets.

All the above is a description of how TCP/IP is defined. It may well be
the case that Microsoft have implemented it differently in Windows -
let's test that. Start with larger responses to see how your client
behaves over multiple packets.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

I've done more investigation at both the HTTP level and TCP level.  The
packets at the HTTP level are virtually identical whether or not I use the
proxy (shouldn't be a big surprise).  However, when I sniff the packets on
the wire, I find that not every TCP packet sent by Pound is ACK'd by the
client.  Correct me if I'm wrong, but every TCP packet needs to be ACK'd,
regardless if Pound split the packets or not, right?  I see, at least on my
sample, that 4 packets went unACK'd, then there was a delay of 20 seconds
(the Client timeout value), then Pound tore down the connections to the
server and client, and then the client rebuilt the connections and finished
loading the page.

Could the missing ACKs be causing the problem I am seeing?  I am using a
Windows package called Observer to sniff the packets.  I am running Pound on
Linux and can create a TCPDUMP file for you, but I'm not sure of the
switches for TCPDUMP to get the information you need.

The missing ACKs are pretty repeatable, if I refresh the page, the missing
ACKs occur in the virtually the same places over and over again.  One place
it's repeatable is when Pound split the HTTP request across multiple
packets, in my example below where Pound split the 200 message from the rest
of the content, could be coincidence.

I hope I'm not barking up the wrong tree, and I'm trying to do the best
troubleshooting I can given my limited knowledge of HTTP and TCP/IP.  I
appreciate your patience and education in getting me up to speed to
troubleshoot such a nagging problem.

Thanks again for your help,
Steven




> -----Original Message-----
> From: Robert Segall [mailto:roseg(at)apsis.ch]
> Sent: Tuesday, July 05, 2005 10:19 AM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> On Tue, 5 Jul 2005 09:56:06 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> wrote:
> 
> > Update - Potential Breakthrough??!!
> >
> > I was able to sniff packets from the proxy to our backend server, and
> from
> > the proxy to the client and found some interesting results.
> >
> > When I ran the application without the Proxy, everything ran fine as
> > suspected, each packet matching what was sent and received.  When I
> ran the
> > application through the Proxy and decoded each packet, I found that
> just
> > before all of the client timeouts start occurring, Pound splits 1
> packet
> > into 2, and I believe that's where the problems start.
> >
> > Below I've attached the original packet from the backend server, and
> then
> > the 2 subsequent packets that Pound creates.  From my simple
> perspective, it
> > seems like the browser gets confused when a packet either doesn't
> measure up
> > to the content length properly, or when a 200 packet with data gets
> split, I
> > really have no idea though..  ;-)  The next packets in the sequence
> are all
> > TCP Fin packets which close the connections after the Client Timeout
> value
> > is reached.  Then the client opens up fresh connections, and finishes
> > loading the page. (This is where I errantly thought the problem had to
> do
> > with IE and 4 connections, I thought they were all used up, but in
> reality,
> > IE is just waiting for something, and then the timeouts hit, and
> everything
> > resets and IE is happy again)
> >
> > Hopefully with this information you can glean some insight into the
> issue
> > and determine the proper course of action.  Thanks for your help, and
> I hope
> > I didn't confuse you too much.
> 
> Sorry, but this isn't it. HTTP works over TCP sockets (aka SOCK_STREAM)
> and thus is never even aware of the TCP packets. It does not matter how
> many packets there are - they are automatically reassembled into a data
> stream before the application ever sees them. The only thing that
> matters is if the data contents (split over however many packets and/or
> fragments) is the same.
> 
> BTW, the Content-length header indicates the HTTP payload size - it has
> nothing to do with the TCP payload.
> 
> Try sending a larger file (anything over 1.5K - that is the normal TCP
> payload limit) and you'll see the HTTP response split over multiple TCP
> packets.
> 
> All the above is a description of how TCP/IP is defined. It may well be
> the case that Microsoft have implemented it differently in Windows -
> let's test that. Start with larger responses to see how your client
> behaves over multiple packets.
> --
> Robert Segall
> Apsis GmbH
> Postfach, Uetikon am See, CH-8707
> Tel: +41-44-920 4904
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000/1120576720000

Update,

If I go into IE's settings and tell it not to use HTTP1.1 at all, the
application works fine without any delay.  It problem seems to happen only
when HTTP1.1 is selected at the browser level.

Hope that helps.

Steven



> -----Original Message-----
> From: Steven Vlach [mailto:Home_Boy(at)msn.com]
> Sent: Wednesday, July 06, 2005 10:17 AM
> To: pound(at)apsis.ch
> Subject: RE: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> Hi Robert,
> 
> I've done more investigation at both the HTTP level and TCP level.  The
> packets at the HTTP level are virtually identical whether or not I use the
> proxy (shouldn't be a big surprise).  However, when I sniff the packets on
> the wire, I find that not every TCP packet sent by Pound is ACK'd by the
> client.  Correct me if I'm wrong, but every TCP packet needs to be ACK'd,
> regardless if Pound split the packets or not, right?  I see, at least on
> my
> sample, that 4 packets went unACK'd, then there was a delay of 20 seconds
> (the Client timeout value), then Pound tore down the connections to the
> server and client, and then the client rebuilt the connections and
> finished
> loading the page.
> 
> Could the missing ACKs be causing the problem I am seeing?  I am using a
> Windows package called Observer to sniff the packets.  I am running Pound
> on
> Linux and can create a TCPDUMP file for you, but I'm not sure of the
> switches for TCPDUMP to get the information you need.
> 
> The missing ACKs are pretty repeatable, if I refresh the page, the missing
> ACKs occur in the virtually the same places over and over again.  One
> place
> it's repeatable is when Pound split the HTTP request across multiple
> packets, in my example below where Pound split the 200 message from the
> rest
> of the content, could be coincidence.
> 
> I hope I'm not barking up the wrong tree, and I'm trying to do the best
> troubleshooting I can given my limited knowledge of HTTP and TCP/IP.  I
> appreciate your patience and education in getting me up to speed to
> troubleshoot such a nagging problem.
> 
> Thanks again for your help,
> Steven
> 
> 
> 
> 
> > -----Original Message-----
> > From: Robert Segall [mailto:roseg(at)apsis.ch]
> > Sent: Tuesday, July 05, 2005 10:19 AM
> > To: pound(at)apsis.ch
> > Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> > and saving attachments
> >
> > On Tue, 5 Jul 2005 09:56:06 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> > wrote:
> >
> > > Update - Potential Breakthrough??!!
> > >
> > > I was able to sniff packets from the proxy to our backend server, and
> > from
> > > the proxy to the client and found some interesting results.
> > >
> > > When I ran the application without the Proxy, everything ran fine as
> > > suspected, each packet matching what was sent and received.  When I
> > ran the
> > > application through the Proxy and decoded each packet, I found that
> > just
> > > before all of the client timeouts start occurring, Pound splits 1
> > packet
> > > into 2, and I believe that's where the problems start.
> > >
> > > Below I've attached the original packet from the backend server, and
> > then
> > > the 2 subsequent packets that Pound creates.  From my simple
> > perspective, it
> > > seems like the browser gets confused when a packet either doesn't
> > measure up
> > > to the content length properly, or when a 200 packet with data gets
> > split, I
> > > really have no idea though..  ;-)  The next packets in the sequence
> > are all
> > > TCP Fin packets which close the connections after the Client Timeout
> > value
> > > is reached.  Then the client opens up fresh connections, and finishes
> > > loading the page. (This is where I errantly thought the problem had to
> > do
> > > with IE and 4 connections, I thought they were all used up, but in
> > reality,
> > > IE is just waiting for something, and then the timeouts hit, and
> > everything
> > > resets and IE is happy again)
> > >
> > > Hopefully with this information you can glean some insight into the
> > issue
> > > and determine the proper course of action.  Thanks for your help, and
> > I hope
> > > I didn't confuse you too much.
> >
> > Sorry, but this isn't it. HTTP works over TCP sockets (aka SOCK_STREAM)
> > and thus is never even aware of the TCP packets. It does not matter how
> > many packets there are - they are automatically reassembled into a data
> > stream before the application ever sees them. The only thing that
> > matters is if the data contents (split over however many packets and/or
> > fragments) is the same.
> >
> > BTW, the Content-length header indicates the HTTP payload size - it has
> > nothing to do with the TCP payload.
> >
> > Try sending a larger file (anything over 1.5K - that is the normal TCP
> > payload limit) and you'll see the HTTP response split over multiple TCP
> > packets.
> >
> > All the above is a description of how TCP/IP is defined. It may well be
> > the case that Microsoft have implemented it differently in Windows -
> > let's test that. Start with larger responses to see how your client
> > behaves over multiple packets.
> > --
> > Robert Segall
> > Apsis GmbH
> > Postfach, Uetikon am See, CH-8707
> > Tel: +41-44-920 4904
> >
> > --
> > To unsubscribe send an email with subject 'unsubscribe' to
> pound(at)apsis.ch.
> > Please contact roseg(at)apsis.ch for questions.
> > http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> > 07/1120571164000/1120576720000
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000/1120663019000

On Wed, 6 Jul 2005 10:16:59 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> Hi Robert,
> 
> I've done more investigation at both the HTTP level and TCP level. 
> The packets at the HTTP level are virtually identical whether or not I
> use the proxy (shouldn't be a big surprise).  However, when I sniff
> the packets on the wire, I find that not every TCP packet sent by
> Pound is ACK'd by the client.  Correct me if I'm wrong, but every TCP
> packet needs to be ACK'd, regardless if Pound split the packets or
> not, right?  I see, at least on my sample, that 4 packets went
> unACK'd, then there was a delay of 20 seconds(the Client timeout
> value), then Pound tore down the connections to the server and client,
> and then the client rebuilt the connections and finished loading the
> page.
> 
> Could the missing ACKs be causing the problem I am seeing?  I am using
> a Windows package called Observer to sniff the packets.  I am running
> Pound on Linux and can create a TCPDUMP file for you, but I'm not sure
> of the switches for TCPDUMP to get the information you need.
> 
> The missing ACKs are pretty repeatable, if I refresh the page, the
> missing ACKs occur in the virtually the same places over and over
> again.  One place it's repeatable is when Pound split the HTTP request
> across multiple packets, in my example below where Pound split the 200
> message from the rest of the content, could be coincidence.
> 
> I hope I'm not barking up the wrong tree, and I'm trying to do the
> best troubleshooting I can given my limited knowledge of HTTP and
> TCP/IP.  I appreciate your patience and education in getting me up to
> speed to troubleshoot such a nagging problem.
> 
> Thanks again for your help,
> Steven

If the ACKs are really missing then you have major networking issues
(nothing to do with Pound or IE - this happens at the TCP stack level).
I suggest you check carefully, as some implementations support sliding
windows, where several ACKs are sent at once.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

On Wed, 6 Jul 2005 16:29:19 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> Update,
> 
> If I go into IE's settings and tell it not to use HTTP1.1 at all, the
> application works fine without any delay.  It problem seems to happen
> only when HTTP1.1 is selected at the browser level.
> 
> Hope that helps.
> 
> Steven

Did you try my suggestion about the two tcpwatch instances? That would
give you the best information about what happens at the HTTP level
(rather than TCP, which can be confusing).
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

I setup 2 instances of tcpwatch, one between the browser and Pound, and the
other between Pound and IIS, and configured the browser and Pound to go
through the tcpwatch's.  The request/responses were identical (with the
exception of the X-Forwarded header from Pound).

Attached are the request/responses from the client to Pound when a refresh
is done on the page in question.  What doesn't show up in this log are the
"timings" of the request/responses.  All of the transactions occur
sub-second, except for the Packet 3 response, where the server closes the
connection between the browser and Pound after 20 seconds, not in the
sub-second range like the other transactions, then the Packet 4 request
triggers.  All of the connections between Pound and IIS close sub-second as
well.

I'm not sure what I'm looking for in each of the request/responses, tcpwatch
doesn't seem to indicate any sort of error.  I hope this information makes
sense to you.

Steven


> -----Original Message-----
> From: Robert Segall [mailto:roseg(at)apsis.ch]
> Sent: Thursday, July 07, 2005 6:29 AM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> On Wed, 6 Jul 2005 16:29:19 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> wrote:
> 
> > Update,
> >
> > If I go into IE's settings and tell it not to use HTTP1.1 at all, the
> > application works fine without any delay.  It problem seems to happen
> > only when HTTP1.1 is selected at the browser level.
> >
> > Hope that helps.
> >
> > Steven
> 
> Did you try my suggestion about the two tcpwatch instances? That would
> give you the best information about what happens at the HTTP level
> (rather than TCP, which can be confusing).
> --
> Robert Segall
> Apsis GmbH
> Postfach, Uetikon am See, CH-8707
> Tel: +41-44-920 4904
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000/1120735747000

On Thu, 7 Jul 2005 13:20:57 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> Hi Robert,
> 
> I setup 2 instances of tcpwatch, one between the browser and Pound,
> and the other between Pound and IIS, and configured the browser and
> Pound to go through the tcpwatch's.  The request/responses were
> identical (with the exception of the X-Forwarded header from Pound).
> 
> Attached are the request/responses from the client to Pound when a
> refresh is done on the page in question.  What doesn't show up in this
> log are the"timings" of the request/responses.  All of the
> transactions occur sub-second, except for the Packet 3 response, where
> the server closes the connection between the browser and Pound after
> 20 seconds, not in the sub-second range like the other transactions,
> then the Packet 4 request triggers.  All of the connections between
> Pound and IIS close sub-second as well.
> 
> I'm not sure what I'm looking for in each of the request/responses,
> tcpwatch doesn't seem to indicate any sort of error.  I hope this
> information makes sense to you.
> 
> Steven

Did you edit the logs in any way? There are some very bizarre things in
there, such as a different number of GETs and responses, responses that
start (or do not end) in the expected way (see leading spaces in an 200
OK, responses that do not end in a CRLF), etc.

Was there any difference between the logs between the client and Pound
and the logs between Pound and the back-end?
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

> -----Original Message-----
> From: Robert Segall [mailto:roseg(at)apsis.ch]
> Sent: Friday, July 08, 2005 8:22 AM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> On Thu, 7 Jul 2005 13:20:57 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
> wrote:
> 
> > Hi Robert,
> >
> > I setup 2 instances of tcpwatch, one between the browser and Pound,
> > and the other between Pound and IIS, and configured the browser and
> > Pound to go through the tcpwatch's.  The request/responses were
> > identical (with the exception of the X-Forwarded header from Pound).
> >
> > Attached are the request/responses from the client to Pound when a
> > refresh is done on the page in question.  What doesn't show up in this
> > log are the"timings" of the request/responses.  All of the
> > transactions occur sub-second, except for the Packet 3 response, where
> > the server closes the connection between the browser and Pound after
> > 20 seconds, not in the sub-second range like the other transactions,
> > then the Packet 4 request triggers.  All of the connections between
> > Pound and IIS close sub-second as well.
> >
> > I'm not sure what I'm looking for in each of the request/responses,
> > tcpwatch doesn't seem to indicate any sort of error.  I hope this
> > information makes sense to you.
> >
> > Steven
> 
> Did you edit the logs in any way? There are some very bizarre things in
> there, such as a different number of GETs and responses, responses that
> start (or do not end) in the expected way (see leading spaces in an 200
> OK, responses that do not end in a CRLF), etc.

I did not edit the files at all in any way.  The messages are exactly as
tcpwatch generated them.  Looking at the asp code which generates these
responses, it doesn't surprise me that they look weird.  If there are
specific things which should be modified to make them better behaved, please
let me know, and I will change what I can.  The software is a commercial
application, but it doesn't surprise me that the code isn't "clean".

> 
> Was there any difference between the logs between the client and Pound
> and the logs between Pound and the back-end?


I used a program called Examdiff Pro to compare the files between the
browser and pound, and pound and IIS, and the only lines it picked up on
were the added X-Forward headers that Pound adds, otherwise, Examdiff found
the files to be identical, spaces, special characters, CRLF and all.




> Robert Segall
> Apsis GmbH
> Postfach, Uetikon am See, CH-8707
> Tel: +41-44-920 4904
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000/1120828921000

I looked again at your transcripts, and googled some. I see that you use
"Cache-control: private" in your responses. Please try replacing that
with "Cache-control: no-cache" and possibly "Pragma: no-cache" (i am
sure you'll find some IIS parameter for that). Some PHP applications
have shown that to be beneficial when talking to IE.

Let me know if this helps.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

> -----Original Message-----
> From: Robert Segall [mailto:roseg(at)apsis.ch]
> Sent: Friday, July 15, 2005 8:37 AM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts, IE
> and saving attachments
> 
> I looked again at your transcripts, and googled some. I see that you use
> "Cache-control: private" in your responses. Please try replacing that
> with "Cache-control: no-cache" and possibly "Pragma: no-cache" (i am
> sure you'll find some IIS parameter for that). Some PHP applications
> have shown that to be beneficial when talking to IE.

I changed the Cache-control to no-cache with no change in results.  I
couldn't force IIS to use the Pragma: no-cache, maybe because it's a HTTP
1.0 command, and IIS is using HTTP 1.1??

Is there a way to force Pound to use HTTP 1.0 to the backend server for HTTP
like there is for HTTPS and IE??  Something like a NoHTTP 1 setting??  If I
force the browser to use HTTP 1.0, the problem goes away, so I figure if I
could tell Pound to use HTTP 1.0 to the backend, that would fix the issue
and eliminate me from configuring every browser to use 1.0.  Just a thought.

Thanks for continuing to look at this problem.

Steven


> 
> Let me know if this helps.
> --
> Robert Segall
> Apsis GmbH
> Postfach, Uetikon am See, CH-8707
> Tel: +41-44-920 4904
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 07/1120571164000/1121434615000

On Mon, 18 Jul 2005 16:08:28 -0500 "Steven Vlach" <Home_Boy(at)msn.com>
wrote:

> 
> 
> > -----Original Message-----
> > From: Robert Segall [mailto:roseg(at)apsis.ch]
> > Sent: Friday, July 15, 2005 8:37 AM
> > To: pound(at)apsis.ch
> > Subject: Re: [Pound Mailing List] Issue with Pound, Client Timeouts,
> > IE and saving attachments
> > 
> > I looked again at your transcripts, and googled some. I see that you
> > use"Cache-control: private" in your responses. Please try replacing
> > that with "Cache-control: no-cache" and possibly "Pragma: no-cache"
> > (i am sure you'll find some IIS parameter for that). Some PHP
> > applications have shown that to be beneficial when talking to IE.
> 
> I changed the Cache-control to no-cache with no change in results.  I
> couldn't force IIS to use the Pragma: no-cache, maybe because it's a
> HTTP 1.0 command, and IIS is using HTTP 1.1??

The Pragma is indeed 1.0

> Is there a way to force Pound to use HTTP 1.0 to the backend server
> for HTTP like there is for HTTPS and IE??  Something like a NoHTTP 1
> setting??  If I force the browser to use HTTP 1.0, the problem goes
> away, so I figure if I could tell Pound to use HTTP 1.0 to the
> backend, that would fix the issue and eliminate me from configuring
> every browser to use 1.0.  Just a thought.

No, there is no way, and I very much doubt it would help.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904