/ Zope / Apsis / Pound Mailing List / Archive / 2006 / 2006-03 / RE: [Pound Mailing List] Bad URL

[ << ] [ >> ]

[ Pound Performance / "Kevin Minnick" ... ] [ NAPT confuses pound / Victor Granic ... ]

RE: [Pound Mailing List] Bad URL
"Dean" <deanmaunder(at)telstra.com>
2006-03-01 06:35:35 [ FULL ]
Thanks Robert,
What I am seeing in the pound log is:
pound: bad request "7628ÿØÿà" from xx.xx.xxx.xx 
My device is trying to post image data, can I turn off the checking for
this?

-----Original Message-----
From: Robert Segall [mailto:roseg(at)apsis.ch] 
Sent: Monday, 27 February 2006 8:30 PM
To: pound(at)apsis.ch
Subject: Re: [Pound Mailing List] Bad URL

On Fri, 2006-02-24 at 07:52 +1000, Dean wrote:[...]

Check your log files - Pound tells you exactly why it rejected the request.
With 2.x we have eliminated the (default) checks on URL, though you can
still specify them if you wish. In your case it's more likely you have a bad
request type (something other than GET, POST or HEAD).
--
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

[...]

RE: [Pound Mailing List] Bad URL
Robert Segall <roseg(at)apsis.ch>
2006-03-06 11:39:25 [ FULL ]
On Wed, 2006-03-01 at 15:35 +1000, Dean wrote:[...]

I can assure you that "7628ÿØÿà" is not a valid HTTP request. We
normally permit stuff like GET, POST, HEAD, etc.

I suggest you check (sniff) your data-stream and find out why such a
request is sent.[...]

Re: [Pound Mailing List] Pound Performance
Robert Segall <roseg(at)apsis.ch>
2006-03-06 11:46:00 [ FULL ]
On Tue, 2006-02-28 at 13:12 -0500, Kevin Minnick wrote:[...]

Many thanks Kevin. These 73 requests/second are very much in line with
our tests and other reports we received.

Keep the reports going.[...]

RE: [Pound Mailing List] Bad URL
"Dean" <deanmaunder(at)telstra.com>
2006-03-06 23:13:11 [ FULL ]
Thanks Robert,
I thought the "7628ÿØÿà" was in the body of the request (which is where is
was supposed to be), once I figured out it WAS the request, everything
became a LOT clearer. :)
Thanks for your help.


-----Original Message-----
From: Robert Segall [mailto:roseg(at)apsis.ch] 
Sent: Monday, 6 March 2006 8:39 PM
To: pound(at)apsis.ch
Subject: RE: [Pound Mailing List] Bad URL

On Wed, 2006-03-01 at 15:35 +1000, Dean wrote:[...]

I can assure you that "7628ÿØÿà" is not a valid HTTP request. We normally
permit stuff like GET, POST, HEAD, etc.

I suggest you check (sniff) your data-stream and find out why such a request
is sent.
--
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

[...]

Re: [Pound Mailing List] Pound Performance
"Agustinus Mulyawan" <a.mulyawan(at)gmail.com>
2006-03-07 12:17:09 [ FULL ]
I have test the pound with web application stress tool,
in the 500 for the "stress level (threads)", and 1 for the "stress
multiplier (sockets per threads)"
I assume it 500 user.
I test it for a while, the pound manage to survive, but some lagging for the
1st open up the site..
Then I try to test with 1000 for the "stress level (threads)", and 1 for the
"stress multiplier (sockets per threads)"
And the web show up this error messages, "503 Service Unavailable"
in the pound log, i found this error (repeatedly)
     pound pound: error copy server cont: Connection reset by peer
     pound pound: error flush headers to 192.168.100.1: Connection reset by
peer
     pound pound: error copy server cont: Connection timed out
     pound pound: backend 192.168.0.251:80 create: Too many open file

I had try to set the clients to 600, and set the ulimit

[root(at)pound root]# ulimit -a
core file size        (blocks, -c) unlimited
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 999999
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) unlimited
cpu time             (seconds, -t) unlimited
max user processes            (-u) unlimited
virtual memory        (kbytes, -v) unlimited

what should I do so I can achieve the best performance ?
do i need configure the sysctl?

On 3/6/06, Robert Segall <roseg(at)apsis.ch> wrote:[...]
Attachments:  
text.html text/html 3789 Bytes

Re: [Pound Mailing List] Pound Performance
Robert Segall <roseg(at)apsis.ch>
2006-03-07 17:49:49 [ FULL ]
On Tue, 2006-03-07 at 11:17 +0000, Agustinus Mulyawan wrote:[...]

At a 1000 concurrent requests you may be running into the pthread limit
of max threads. You'd need to recompile the library to go beyond that.

Given your log messages you may also have a problem with your back-ends:
are you sure they can actually deal with this sort of activity?

Do you actually expect that sort of load on your site?[...]

Re: [Pound Mailing List] Pound Performance
"Agustinus Mulyawan" <a.mulyawan(at)gmail.com>
2006-03-08 04:29:13 [ FULL ]
Yes, the site have many concurent user.
Even more then 1000 concurent user, the bandwith usage for web up to 4 Mbps
(mainly text display)
I just want to test it before implement the pound for load balance.
The backend work fine, when heavily load with many user, pound not respond.

After send the mail, I try to recompile pound with the new configuration of
"ulimit"
And I test with 1000 for the "stress level (threads)", and 1 for the
"stress multiplier (sockets per threads)"
I open the test site, and the pound still manage to handle the request.
I will do some more testing, and check the error log in the pound.

But I have something in my mind, Is the effect of reconfigure the ulimit,
may result after recompile the pound?

And about this, "You'd need to recompile the library to go beyond that" --->
Which library I should recompile? Could you explain me a little bit?
And what linux OS I should use?

Thanks..


On 3/7/06, Robert Segall <roseg(at)apsis.ch> wrote:[...]
Attachments:  
text.html text/html 5040 Bytes

Re: [Pound Mailing List] Pound Performance
"T.Nagahama" <nag(at)ash.jp>
2006-03-08 09:38:48 [ FULL ]
Hello.
[...]

I feel ...

a. How many servers in this test ?
"1000 user test" needs 2 or 3 back-ends.
(Pentium 1GHz class, test with Apatch2.0 and ab.)

b. How config your pound ?
pound needs "RewriteRedirect 0", if with many users.
(at pound 1.9.x)

c. "ulimit" use at execute, not at compile.
I use below. pound can make 1800 threads. (my record)

----start
#!/bin/sh --

ulimit -n 4000
ulimit -u 2000

/usr/local/pound/pound -f /usr/local/pound/pound.cfg 2>&1
-----end

Re: [Pound Mailing List] Pound Performance
Robert Segall <roseg(at)apsis.ch>
2006-03-08 12:41:10 [ FULL ]
On Wed, 2006-03-08 at 03:29 +0000, Agustinus Mulyawan wrote:[...]

There is no way to get 1000 concurrent requests over a 4Mbps link.
Simple calculation:

- assume full-duplex link.
- assume (optimistically) that 4Mbps translates into 400KBytes/second
- assume (optimistically) that a single response amounts to 2K

Result: max. 200 requests/second to fully saturate your link.

I strongly suggest you do your homework before you try to set up a
production server, and never confuse the number of concurrent (aka
on-line) users with the number of concurrent requests.
[...]

ulimit has nothing to do with compilation. Setting a new ulimit affects
all processes you run from now on from the same shell. Please read the
man page for more details.
[...]

libpthread
[...]

Before you choose one I suggest you learn some more about how to
configure them - or, even better, hire somebody to do a professional job
for you.[...]

Re: [Pound Mailing List] Pound Performance
"Agustinus Mulyawan" <a.mulyawan(at)gmail.com>
2006-03-08 12:57:25 [ FULL ]
I have tested the pound again.. now with other test tool...
I assume the web stress tool from microsoft somehow does't work well.
Traffic with 1000 user so small, for each server it only reach out 300kbps.
With this new test tool with 520 concurent user,
the backend server hang, the IIS thread error.
And the traffic from each server reach 500kbps.
Total Traffic around 1.5 Mbps for 520 concurrent user.
Current Backend server I use it's 1.8 Ghz processor with 1 Gb RAM, with 2
VMWare Server (so it will look like I had 3 web server)
And the Pound use 1.8 Ghz processor with 512 Mb RAM.
Tomorrow, I will try to test to the real webserver as the backend.

The real server of the website use 4 web server, with 2 pcs 2.8 Ghz xeon
processor and 2 Gb RAM.
Currently not using any load balancer, but redirect traffic for specific
user to other server, it's not optimize bcause you never know for sure which
one of the user heavily load the traffic to the web server.
For some reason, hardware load balancer failed (F5 Big IP).
So I search on the web, and found this great balancer pound.
It work for the condition in our site with some minor adjustment to the web
pages, so I want to implement the pound as load balancer.
But before do that I need to test the load balancer first.

Does any one can give me suggestion what kind of Linux OS that I should use
for pound?
How about to recompile the library, what library I should recompile?
I'm not really know about linux, thanks.


On 3/8/06, T.Nagahama <nag(at)ash.jp> wrote:[...]
Attachments:  
text.html text/html 3260 Bytes

Re: [Pound Mailing List] Pound Performance
"Agustinus Mulyawan" <a.mulyawan(at)gmail.com>
2006-03-08 13:51:19 [ FULL ]
On 3/8/06, Robert Segall <roseg(at)apsis.ch> wrote:
[...]


Thanks I notice it now, after test with other stress tool... That I mention
from previous mail.
Web stress tool of Microsoft --> Traffic with 1000 user so small, for each
server 300kbps. (900kbps)
New stress tool --> 520 User traffic 500kbps (total around 1.5 Mbps)

[...]


Thanks, I'm doing the homework already. And thanks for the explanation of
urs, from the reading from microsoft web stress tool confusing enough
max total 900 Kbps for 1000 user, the concurent connection around 36
concurent request, for each around 10 request

I found out from the new stress tool with 520 user, around 60 concurent
request for each around 20 request for each server before it crash.
The backend with VMWare can not support 520 user, absolutely no way it can
manage 1000 user that I test earlier.

The production server never been tested before, I'm just test it for
efficiency in the future.
Of course I need learn more about the production server, I'm new with this
sort of things.


[...]


That's why I ask u about that to make sure, the microsoft stress tool that I
install in other pc somehow got error.
The result of the stress test no good.
First it got error, later on the result doesn't give any error message.
Later on I try to find other stress tool, and install it in other pc.
I'm no expert with linux, I learn more with this. Thanks again.
[...]


I'm going to read more about that, thanks.
[...]


I just currious if there is other Linux OS that really suitable for pound,
that no need too much configuration.
"hire somebody to do a professional job for you." --> some hardware load
balancer technician had try to configure the load balance of the web, but no
result bcause of several things.

I'll do that learning thing, somehow you make me want to know more about
linux.
I ask because I do not really good with linux, sorry for stupid thing I ask
u..
Thanks for your explanation and for pound, it is really a good load
balancer...

--[...]
Attachments:  
text.html text/html 5878 Bytes

RE: [Pound Mailing List] Pound for apache on Windows 2003
"Joe Gooch" <mrwizard(at)k12system.com>
2006-03-10 21:14:05 [ FULL ]
I have compiled Pound 1.9.x on Cygwin with little to no difficulty.
(then again, I know C)  I haven't done any extensive testing to
determine performance in such a configuration, it was mainly for SSL
offloading and a development server.

It should work fine on windows.  I don't have any benchmarks but it
should work for you.

Joe

[...]
under[...]
pound(at)apsis.ch.[...]

Re: [Pound Mailing List] Type PARM
Robert Segall <roseg(at)apsis.ch>
2006-03-13 12:51:40 [ FULL ]
On Sun, 2006-03-12 at 22:26 -0600, Russell Valentine wrote:[...]

This is NOT correct behaviour - if it happens. Look at the code: one
would expect a 503 and not a 500 if an error of that kind were to occur.

I suggest you have a look at your logs and see what exactly the reason
for the 500 was.

Let us know.[...]

Re: [Pound Mailing List] Type PARM
Robert Segall <roseg(at)apsis.ch>
2006-03-13 13:50:08 [ FULL ]
On Sun, 2006-03-12 at 22:26 -0600, Russell Valentine wrote:[...]

On second thought there is a problem with the PARM code: try modifying
the code in svc.c, line 617 and on to read:

    case SESS_PARM:
        if(get_REQUEST(key, svc, request)) {
            if((sp = sess_find(svc->sessions, key)) == NULL) {
                /* no session yet - create one */
                res = rand_backend(svc->backends, random()
%svc->tot_pri);
                svc->sessions = sess_add(svc->sessions, key, res);
            } else
                res = sp->to_host;
ADD:    } else {
ADD:        res = rand_backend(svc->backends, random() % svc->tot_pri);
        }
        break;

(just add the two lines marked that way). Recompile and it should work.[...]

Re: [Pound Mailing List] pound & https
"Simon Matter" <simon.matter(at)ch.sauter-bc.com>
2006-03-14 12:45:49 [ FULL ]
> Hi,[...]

Pound always speaks cleartext to the backends. If you need this traffic
encrypted, you have to use stunnel or similar.

Simon
[...]

Re: [Pound Mailing List] pound & https
lars <lists(at)sumpfralle.de>
2006-03-15 00:12:40 [ FULL ]
Hi,
[...]
I solved the problem by using the following directives for apache:
UseCanonicalName on
ServerName INTERNAL.NAME
(the INTERNAL.NAME should be the one, that is used in the pound config file)

It is important, that you put these statements near to the beginning of
your apache config file.

regards,
Lars

Re: [Pound Mailing List] Type PARM
Russell Valentine <russ(at)altec.org>
2006-03-15 03:43:40 [ FULL ]
Robert Segall said the following on 03/13/06 06:50:[...][...][...]

Alright thanks!

RE: [Pound Mailing List] Pound for apache on Windows 2003
"Joe Gooch" <mrwizard(at)k12system.com>
2006-03-15 15:28:49 [ FULL ]
Pound 2.0.2 compiles on cygwin with no modifications.

If you copy cygcrypto-*.dll, cygssl-*.dll, and cygwin1.dll to a
directory with the pound.exe, you should be able to run pound without
installing a full cygwin.  If it complains about additional dlls, copy
them into the same directory.


Joseph Gooch
Sapphire Suite Product Manger
K12 Systems, Inc.
(866) 366-9540

[...]
on[...]
pound(at)apsis.ch.[...]

Re: [Pound Mailing List] Pound and webmin
Christian Sell <christian.sell(at)netcologne.de>
2006-03-15 21:31:42 [ FULL ]
I dont think you can deploy anything "in" pound. Its not a web server, its a
proxy.


Zitat von Brett Hall <brett.hall(at)sirsidynix.com>:
[...]

------------------
Christian Sell

RE: [Pound Mailing List] Pound and webmin
"Brett Hall" <brett.hall(at)sirsidynix.com>
2006-03-15 21:37:48 [ FULL ]
I realize that...my mistake.  Please read this as;

Has anyone had experience handling usermin and/or webmin requests using
pound.  We have several apps distributed through pound so we know how to
use it and its purpose.

-----Original Message-----
From: Christian Sell [mailto:christian.sell(at)netcologne.de] 
Sent: Wednesday, March 15, 2006 2:32 PM
To: pound(at)apsis.ch
Subject: Re: [Pound Mailing List] Pound and webmin

I dont think you can deploy anything "in" pound. Its not a web server,
its a
proxy.


Zitat von Brett Hall <brett.hall(at)sirsidynix.com>:
[...]
pound(at)apsis.ch.[...]

------------------
Christian Sell
[...]

Re: [Pound Mailing List] Debian Install
"Sam Johnston" <samjie(at)gmail.com>
2006-03-23 22:42:19 [ FULL ]
User "user_name"
              Specify  the  user  Pound  will  run  as  (must  be  defined  in
              /etc/passwd).

It looks like it doesn't like the directive itself... group membership
doesn't matter.

Sam

On 3/21/06, Sean Minotti <sminotti(at)nycap.rr.com> wrote:[...]

Re: [Pound Mailing List] Debian Install
Colin Hogben <pound(at)pythontech.co.uk>
2006-03-24 09:23:34 [ FULL ]
Sam Johnston wrote:
[...]
[...]

I looks like pound 2.* requires quotes around the user name, whereas 
pound 1.* requires no quotes.
[...]

Re: [Pound Mailing List] Debian Install
"Norbert Klamann" <Norbert.Klamann(at)klamann-software.de>
2006-03-24 10:02:12 [ FULL ]
Am 24.3.2006 schrieb "Colin Hogben" <pound(at)pythontech.co.uk>:
[...][...][...]

But whats about 'WebDav 1' . This soes not work under these
circumstances.


Norbert

Re: [Pound Mailing List] Debian Install
"Klaus Alexander Seistrup" <kseistrup(at)gmail.com>
2006-03-24 11:23:22 [ FULL ]
Norbert Klamann wrote:
[...]

»WebDAV 1« works for me in v2.0.2, but you might have to configure
pound with '--enable-msdav' in order to get the functionality you're
looking for.

Cheers,

--
Klaus Alexander Seistrup
Copenhagen · Denmark
http://minji.szn.dk/

Re: [Pound Mailing List] Debian Install
"Norbert Klamann" <Norbert.Klamann(at)klamann-software.de>
2006-03-24 12:15:02 [ FULL ]
Am 24.3.2006 schrieb "Klaus Alexander Seistrup" <kseistrup(at)gmail.com>:
[...][...][...]

At the moment I have my shar of problems with the syntax itself. I
personally have a feel that the problem is debian-related. My box was
automatically upgraded recently.

Cheers

Norber

Re: [Pound Mailing List] Re: "no back-end"
Eric McCarthy <eric(at)desert.net>
2006-03-26 06:57:22 [ FULL ]
We've noticed something similar, but it wasn't pound's fault. We use the pf
firewall in FreeBSD. The server would hit the maximum number of entries in
the state table and start denying the creation of new connections (incoming
and outgoing). It would fool pound into thinking that all the backends were
down. The solution was to increase max number of state table entries or
lower the expiration time for closed connections.

Hope this helps.

-Eric


On Sun, Mar 26, 2006 at 05:19:47AM +0200, Tobias Brox wrote:[...]

Re: [Pound Mailing List] Re: "no back-end"
Tobias Brox <tobias(at)nordicbet.com>
2006-03-26 07:13:42 [ FULL ]
[Eric McCarthy - Sat at 09:57:22PM -0700][...]

But in such a case, you would see corresponding lines, like:

backend 127.0.0.1:8888 connect: Connection refused

We have one service and multiple backends.  One of the backend was down 14
times through the log file I'm looking through, so pound cannot have
observed an "all servers beeing down"-scenario more than 14 times.  The "no
backend"-message is shown 1691 times during the same log.  Adding up all
"connection refused" and "connection timed out", there are "only" 1242
failures, not enough to explain "all servers down" 1691 times.
[...]

Re: [Pound Mailing List] Re: "no back-end"
Eric McCarthy <eric(at)desert.net>
2006-03-26 07:37:06 [ FULL ]
On Sun, Mar 26, 2006 at 07:13:42AM +0200, Tobias Brox wrote:[...]

Actually, we didn't. It wasn't clear in my previous email, but the pf
firewall was running on the same server as pound. So when the state table
was full, and pound attempted to create a new connection to a backend, the
call to connect would return an error along the lines of "permission
denied". It seemed very bizarre at first, but it makes sense now.

-Eric

Re: [Pound Mailing List] Backend Timeout
Robert Segall <roseg(at)apsis.ch>
2006-03-30 11:51:57 [ FULL ]
On Sun, 2006-03-26 at 05:53 +0200, Tobias Brox wrote:[...]

A back-end is declared to be dead if a connection attempt to it fails
within the defined time-out. It stays dead until resurrected by the
checking thread. This explanation does not account for the HA option.

This means that a server may be dead over quite some time, thus a lot of
requests may fail with "no back-end" for a single back-end failure.

I don't think it would be right to declare a back-end as dead for a
time-out in the middle of a reply. If we just abort this one request
others stand a better chance of getting a reply from an overloaded
server.

Judging by your past messages I suggest you experiment with the various
time-out parameters and see if it helps.[...]

Re: [Pound Mailing List] Pound question..
Robert Segall <roseg(at)apsis.ch>
2006-03-30 12:12:50 [ FULL ]
On Thu, 2006-03-23 at 09:05 -0800, Rick Flower wrote:[...]

No. Pound does not support URL rewriting.[...]

Re: [Pound Mailing List] Backend Timeout
Tobias Brox <tobias(at)nordicbet.com>
2006-03-30 16:15:14 [ FULL ]
[Robert Segall - Thu at 11:51:57AM +0200][...]

Hm, you are of course quite right about this - i.e., a server error can
cause a particular request to freeze, or a particular request can take long
time to process, and it's right to assume the backend is multi-threaded and
capable to handle other requests regardless of the single frozen request.  I
suppose the best option is to try to modify the backend to not accept
further connections when it's trashed.
[...]

Re: [Pound Mailing List] error in pound?
Robert Segall <roseg(at)apsis.ch>
2006-03-31 15:34:26 [ FULL ]
On Fri, 2006-03-31 at 00:53 +0200, Corin Langosch wrote:[...]

"Connection reset by peer" means the other side has closed the
pipe/socket while we were waiting for something. Usually a symptom of
browser being closed (or the user clicking "Stop") while we still had
data to send. I wouldn't worry too much about those.[...]

Re[2]: [Pound Mailing List] error in pound?
Corin Langosch <corinl(at)gmx.de>
2006-03-31 17:11:06 [ FULL ]
Hi!

thanks for your answer. i know specifically these errors are not so bad, but
the
ones that make me worry are those "error copy server cont: Connection reset by
peer" errors.

please correct if i'm wrong, but i currently think

1. "error copy chunk cont: Connection reset by peer" means that the copy from
pound to the client failed. so the client disconnted, aborted or whatever -
this
is not bad.

2. "error copy server cont: Connection reset by peer" means that the copy from
the backend to pound failed. so the webserver must be crashed or pound has an
error - this is bad.

if this is correct, i think i must have some segfaults in my apache.log for
case 2. if there aren't any, there must be something wrong with pound as
everything
with apache/php works fine?

regards,
corin

Friday, March 31, 2006, 3:34:26 PM, you wrote:
RS> On Fri, 2006-03-31 at 00:53 +0200, Corin Langosch wrote:[...]

RS> "Connection reset by peer" means the other side has closed the
RS> pipe/socket while we were waiting for something. Usually a symptom of
RS> browser being closed (or the user clicking "Stop") while we still had
RS> data to send. I wouldn't worry too much about those.
RS> -- 
RS> Robert Segall
RS> Apsis GmbH
RS> Postfach, Uetikon am See, CH-8707
RS> Tel: +41-44-920 4904

Re: Re[2]: [Pound Mailing List] error in pound?
Robert Segall <roseg(at)apsis.ch>
2006-03-31 18:41:10 [ FULL ]
On Fri, 2006-03-31 at 17:11 +0200, Corin Langosch wrote:[...]

Wrong assumption. The two messages are just (detailed) reports about
which part of Pound noticed the problem - but the problem is identical.
Which message gets triggered depends on the response type (if you have
content-length or transfer-encoding: chunked). In both cases the client
closed the socket prematurely.[...]

MailBoxer