Unfortunately, there is no silver bullet. Some disruption will always
occur.

- if you kill the child you loose the requests being currently
processed - i.e. the ones that are in transit. This was the original
idea.

- if you wait until they complete you'll probably loose new requests.
The accept queue on sockets is of limited length, so on a medium site
you'll get failures in the connect.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

On Wed, 31 Aug 2005 10:41:31 -0400 Ed R Zahurak
<ezahurak(at)atlanticbb.net> wrote:

> Robert,
> 
> Just thinking out loud here.  How about allowing for a graceful
> shutdown of an instance of pound?  You could then cascade a few
> instances of pound running behind one main instance, (with minimal
> configuration, that hopefully you'd not have to stop often,) like
> this:
> 
>                          [pound 'overlord']
>                             |          |
>                       [pound 'sub1'] [pound 'sub2']
>                          |     |           |     |
>                       [backend servers all go here.]
> 
> When a configuration on sub1 and sub2 need changed, you could issue a 
> graceful shutdown to those instances, one at a time, to handle current
> connections, not take any new ones, and shut down when the last
> current connection terminates.  The HA already present in the overlord
> process would route new connections to the other subinstance, and when
> the shut-down subinstance is restarted, start routing connections to
> that instance.  The process could then be repeated for the other
> subinstance.
> 
> Obviously, this wouldn't work in a scenario where sessions *must* go
> to one backend box for the life of the session, but I can't think of 
> practical means, aside from saving session information to disk to be 
> read on restart that would solve that, not that I think it's a
> priority to solve that particular problem in the first place.

This is the sort of solution you could implement right now with three
Pound instances. As the master sees that one of the sub-servers is
off-line (because it's reloading its configuration) it would be
considered dead and all requests would go to the second server. When it
comes back on-line with the new configuration it gets resurrected and
then you can take the second one off-line for reconfiguration.

However this only works because you have separate listening sockets. As
a monolithic solutions this fails.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

I would rather have the requests die immediately, relying on the user to
try again.  And when that happens, the supervisor running the new config
with the old sessions (having been restored) will just create another
child and proceed from there.

It's no worse than killing pound completely, which is what I have to do
now.  It's certainly better than disrupting current running requests AND
making everyone's session restart.

Joe

-----Original Message-----
From: Robert Segall [mailto:roseg(at)apsis.ch] 
Sent: Thursday, September 01, 2005 9:18 AM
To: pound(at)apsis.ch
Subject: Re: [Pound Mailing List] changing conf without restart

Unfortunately, there is no silver bullet. Some disruption will always
occur.

- if you kill the child you loose the requests being currently
processed - i.e. the ones that are in transit. This was the original
idea.

- if you wait until they complete you'll probably loose new requests.
The accept queue on sockets is of limited length, so on a medium site
you'll get failures in the connect.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

-- 
To unsubscribe send an email with subject 'unsubscribe' to
pound(at)apsis.ch.
Please contact roseg(at)apsis.ch for questions.
http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-09/1125
580698000

Robert Segall wrote:
> Unfortunately, there is no silver bullet. Some disruption will always
> occur.
> 
> - if you kill the child you loose the requests being currently
> processed - i.e. the ones that are in transit. This was the original
> idea.
> 
> - if you wait until they complete you'll probably loose new requests.
> The accept queue on sockets is of limited length, so on a medium site
> you'll get failures in the connect.

Why can't we have someone accepting and queuing the requests until
the workers complete?

Joe Gooch wrote:
> I would rather have the requests die immediately, relying on the user to
> try again.  And when that happens, the supervisor running the new config
> with the old sessions (having been restored) will just create another
> child and proceed from there.
> 
> It's no worse than killing pound completely, which is what I have to do
> now.  It's certainly better than disrupting current running requests AND
> making everyone's session restart.

But it's not better than just running straight Apache and giving it a
graceful restart, which seems capable of both reloading the config and
not disrupting any requests. I'd really like to retain that behavior
when introducing Pound to the mix.

And esp. in ecommerce applications, having a request killed mid-stream
while your CC data is in transit does not give you a warm feeling.

Maurice

On Thu, 01 Sep 2005 13:27:12 -0600 Maurice Aubrey <maurice(at)redweek.com>
wrote:

> Robert Segall wrote:
> > Unfortunately, there is no silver bullet. Some disruption will
> > always occur.
> > 
> > - if you kill the child you loose the requests being currently
> > processed - i.e. the ones that are in transit. This was the original
> > idea.
> > 
> > - if you wait until they complete you'll probably loose new
> > requests. The accept queue on sockets is of limited length, so on a
> > medium site you'll get failures in the connect.
> 
> Why can't we have someone accepting and queuing the requests until
> the workers complete?

Are you volunteering?

Have a look at the code - there are no queues at all. Each connection is
accepted and immediately passed to a new thread. You stop doing that and
you loose connections.

Keep in mind that there is a single socket - and that can't be changed.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

When I get my head above water, I might do just that.

What about encapsulating the global variables (aka make a pound 
"object") and then when the signal is received

a) instantiate a new pound object from the new config file
b) tell the old pound object to stop accepting connections
c) copy session data from the old to the new pound object
c) tell the new pound object to start accepting all connections
d) 30 seconds later, reap and deallocate the old pound object on the 
theory that it would have
handled all outstanding work and spawned any necessary threads by then.

This approach will finish all outstanding requests because it doesn't 
kill any threads and keeps all live sessions alive.

It is bad because the single pound instance has to be able to read its 
own config file (no big deal... just put the config file in jail with 
pound) and because it requires that the C code be restructured a bit 
more than one might want.  The flip side is that global variables are 
evil in the first instance anyway.

Robert Segall wrote:

> Are you volunteering?
>
>Have a look at the code - there are no queues at all. Each connection is
>accepted and immediately passed to a new thread. You stop doing that and
>you loose connections.
>
>Keep in mind that there is a single socket - and that can't be changed.
>  
>


-- 
Ted Dunning
Chief Scientist
Veoh Networks

I figured this out.

For posterity...

As I suspected, the problem was DNS related.  Looking at the source code 
reveals that pound does, indeed, do a DNS lookup (gethostbyname) when 
doing redirectRewriting.

Meaning that if your local DNS server is not setup to properly resolve 
(as ours was not) then redirectRewrites won't work properly in all cases.

More problematically if the DNS server is not setup properly (pointed to 
a dead DNS server, say) then pound will "hang" on ALL "Redirect" 
response types while it waits for the resolver to timeout.  That's not a 
huge "hang time" but it can be on the order of a second or two. 
Certainly noticable.

One might ask why we didn't have our DNS resolver setup properly in the 
first place.  I can only offer that this is a linux firewall box running 
an absolutely minimal configuration, basically just iptables and pound. 
  iptables doesn't need the resolver and we had just assumed pound 
wouldn't either.  That's the problem with assumptions: makes an ass out 
of you and umptions. :-)

Might I suggest that we modify the docs to indicate that pound relies on 
a properly configured DNS resolver?

Even better (but undoubtably waaay more work), might I suggest that 
pound be modified to *not* rely on the resolver?

It seems, at first blush, that it would be possible for pound to collect 
enough info in the back-end config to eliminate the need to do 
gethostbyname calls.  Like having the back-end config-info contain both 
an IP address *and* the DNS name, for instance.  This would allow the 
"is_be" routine to do a match against returned LOCATION headers without 
having to do a lookup on the LOCATION name.  You'd already have 
everything you need for the comparison without having to do that lookup.

One might also make the argument that eliminating dns lookups would 
speed up pound.  The Apache web-server, after-all, has a disable-lookups 
feature for exactly this reason (performance).

But looking through the code it appears that dns-lookups are only done 
when dealing with redirects, when processing the config file, and at 
initial startup (when binding the listeners).  In all cases, not code 
that gets executed very often, relatively speaking.

So maybe just a note in the docs indicating that pound relies on a 
properly config'd dns resolver is enough.

- Gary


glists(at)greywether.com wrote:
> We're having a problem with a java webapp that sits behind our pound proxy.
> 
> All features of the site work fine when accessed via standard http.  But 
> when accessed via https, redirects returned from various pages fail.
> 
> The login page, for example works when accessed via http and not via 
> https.  The login page's response is a redirect.
> 
> Here's the CURL for both a successfull HTTP request and a unsuccessful 
> HTTPS request for a POST to the login form (a login attempt).  The 
> resulting response (for either a valid or invalid login) is a redirect:
> 
> 
> HTTP (this works)
> -----------------
>  > curl -i -d 
> "emailAddress=gweb(at)greywether.com&clearTextPassword=yw84f8" 
> http://www.expeditiontea.com/karta/login_ProcessForm.action
> 
> HTTP/1.1 302 Moved Temporarily
> Set-Cookie: JSESSIONID=EFF5FB4761150F57AB560115694B4E5B; Path=/karta
> Location: 
>
http://www.expeditiontea.com/karta/productCategories_Show.action;jsessionid=EFF5FB4761150F57AB560115694B4E5B

> 
> Content-Length: 0
> Date: Thu, 01 Sep 2005 01:49:21 GMT
> Server: Apache-Coyote/1.1
> 
> 
> HTTPS (this fails)
> ------------------
>  > curl -i -d "emailAddress=examp(at)greywether.com&clearTextPassword=pw" 
> https://www.expeditiontea.com/karta/login_ProcessForm.action
> 
> HTTP/1.1 302 Moved Temporarily
> Set-Cookie: JSESSIONID=439D653BD4972F695CB1A7A499A09AF4; Path=/karta
> Location: 
>
http://www.expeditiontea.com:443/karta/productCategories_Show.action;jsessionid=439D653BD4972F695CB1A7A499A09AF4

> 
> Content-Length: 0
> Date: Thu, 01 Sep 2005 01:45:51 GMT
> Server: Apache-Coyote/1.1
> 
> 
> (Note that the RewriteRedirect feature has no effect on the above.  I 
> get the same response with it either On or off (1 or 0).  The above was 
> captured with RewriteRedirect off (0).)
> 
> ----
> 
> The problem in the above is obvious.  The "location" when it gets back 
> to the browser (or in this case curl) is written as:
> 
>   http://www.expeditiontea.com:443...
> 
> That's a standard http request onthe https port (443).  Bad.  Seems to 
> me the redirect "location" should be written as:
> 
>   https://www.expeditiontea.com...
> 
> ----
> 
> One other item of note is that with RewriteRedirect "on" the post 
> requests take a relatively long time to process (2 seconds or so 
> compared to "instantaneous" with it off).  This leaves me wondering if 
> pound is doing some sort of dns resolve when it tries to rewrite 
> redirects and, since our internal dns isn't setup for this server, that 
> would fail.  So maybe that's the whole problem?  I need to turn on 
> RewriteRedirect but get reverse-dns working for the local servers IP?
> 
> Any help you can provide is, as always, most appreciated.
> 
> Thanks!
> 
> - Gary
> 
> P.S. Pound config file follows:
> 
> ######################################################################
> ## global options:
> 
> User            www-data
> Group           www-data
> #RootJail       /chroot/pound
> 
> ## allow PUT and DELETE also (by default only GET, POST and HEAD)?:
> ExtendedHTTP    1
> WebDAV          1
> 
> ## Logging: (goes to syslog by default)
> ##      0       no logging
> ##      1       normal
> ##      2       extended
> ##      3       Apache-style (common log format)
> LogLevel        2
> 
> ## check backend every X secs:
> Alive           30
> 
> ## use hardware-accelleration card supported by openssl(1):
> #SSLEngine      <hw>
> 
> ## disable pound's redirect rewriting
> RewriteRedirect 0
> 
> ######################################################################
> ## listen, redirect and ... to:
> 
> ## redirect all requests on port 8888 ("ListenHTTP") to the local 
> webserver see "UrlGroup" below):
> ListenHTTP 207.118.16.116,80
> ListenHTTPS 207.118.16.116,443 /etc/pound/www.expeditiontea.com.pem
> 
> ##
> UrlGroup ".*"
> HeadRequire Host ".*projects.greywether.com.*"
> BackEnd 192.168.73.2,80,1
> EndGroup
> 
> UrlGroup ".*"
> HeadRequire Host ".*www.expeditiontea.com.*"
> BackEnd 192.168.73.4,8080,1
> EndGroup
> 
> UrlGroup ".*"
> BackEnd 192.168.73.6,80,1
> EndGroup
> 

On Fri, 02 Sep 2005 18:02:04 -0700 glists(at)greywether.com wrote:

> I figured this out.
> 
> For posterity...
> 
> As I suspected, the problem was DNS related.  Looking at the source
> code reveals that pound does, indeed, do a DNS lookup (gethostbyname)
> when doing redirectRewriting.
> 
> Meaning that if your local DNS server is not setup to properly resolve
> (as ours was not) then redirectRewrites won't work properly in all
> cases.
> 
> More problematically if the DNS server is not setup properly (pointed
> to a dead DNS server, say) then pound will "hang" on ALL "Redirect" 
> response types while it waits for the resolver to timeout.  That's not
> a huge "hang time" but it can be on the order of a second or two. 
> Certainly noticable.
> 
> One might ask why we didn't have our DNS resolver setup properly in
> the first place.  I can only offer that this is a linux firewall box
> running an absolutely minimal configuration, basically just iptables
> and pound. 
>   iptables doesn't need the resolver and we had just assumed pound 
> wouldn't either.  That's the problem with assumptions: makes an ass
> out of you and umptions. :-)
> 
> Might I suggest that we modify the docs to indicate that pound relies
> on a properly configured DNS resolver?
> 
> Even better (but undoubtably waaay more work), might I suggest that 
> pound be modified to *not* rely on the resolver?
> 
> It seems, at first blush, that it would be possible for pound to
> collect enough info in the back-end config to eliminate the need to do
> gethostbyname calls.  Like having the back-end config-info contain
> both an IP address *and* the DNS name, for instance.  This would allow
> the "is_be" routine to do a match against returned LOCATION headers
> without having to do a lookup on the LOCATION name.  You'd already
> have everything you need for the comparison without having to do that
> lookup.
> 
> One might also make the argument that eliminating dns lookups would 
> speed up pound.  The Apache web-server, after-all, has a
> disable-lookups feature for exactly this reason (performance).
> 
> But looking through the code it appears that dns-lookups are only done
> when dealing with redirects, when processing the config file, and at 
> initial startup (when binding the listeners).  In all cases, not code 
> that gets executed very often, relatively speaking.
> 
> So maybe just a note in the docs indicating that pound relies on a 
> properly config'd dns resolver is enough.

Pound needs to look names up. This is unavoidable if you have virtual
hosts - there is no other way to check if you need to change (rewrite)
the Location header.

You don't need a full DNS, but make sure your names are resolvable. A
simple entry in /etc/hosts with the address and alternate names will do
fine. On newer Linux systems (with resolv+) you can modify
/etc/host.conf to only look for the file, on older systems you can do
that in /etc/resolv.conf.

Thanks for the tip re. docs - we'll do that.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

On Fri, 16 Sep 2005 13:47:26 +0100 Will Tatam <wtatam(at)premierit.com>
wrote:

> Can please someone post a compete syslog.conf they have setup for 
> logging with pound ?
> 
> I have only manged to always end up with everything only going to 
> /var/log/messages and nothing to my /var/log/pound.log or nothing
> being logged to either
> 

That depends pretty much on which syslog you use (Linux and Solaris are
different beasts) and how you compiled Pound.

The way we usually do it:

1. compile Pound so it uses LOCAL7 for logging:

./configure --with-log=LOG_LOCAL7 ...

compile, install, configure.

2. define where the messages for local7 should go. In /etc/syslog.conf
(assuming Linux-style syslog):

local7.=info                    /var/log/pound.log
local7.notice                   /var/log/pound.err

so that normal traffic goes to /var/log/pound.log and everything else
goes to /var/log/pound.err

3. make sure it goes nowhere else. For example

*.*;auth,authpriv,local7.none   -/var/log/syslog

(just look for *.something and add local7.none).

Don't forget to tell syslog about the new configuration.

Hope this helps some.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

On Fri, 2005-09-16 at 17:36 +0200, Robert Segall wrote:

> On Fri, 16 Sep 2005 13:47:26 +0100 Will Tatam <wtatam(at)premierit.com>
> wrote:
> 
> > Can please someone post a compete syslog.conf they have setup for 
> > logging with pound ?
> > 
> > I have only manged to always end up with everything only going to 
> > /var/log/messages and nothing to my /var/log/pound.log or nothing
> > being logged to either
> > 
> 
> That depends pretty much on which syslog you use (Linux and Solaris are
> different beasts) and how you compiled Pound.
> 
> The way we usually do it:
> 
> 1. compile Pound so it uses LOCAL7 for logging:
> 
> ./configure --with-log=LOG_LOCAL7 ...
> 
> compile, install, configure.
> 
> 2. define where the messages for local7 should go. In /etc/syslog.conf
> (assuming Linux-style syslog):
> 
> local7.=info                    /var/log/pound.log
> local7.notice                   /var/log/pound.err
> 
> so that normal traffic goes to /var/log/pound.log and everything else
> goes to /var/log/pound.err
> 
> 3. make sure it goes nowhere else. For example
> 
> *.*;auth,authpriv,local7.none   -/var/log/syslog
> 
> (just look for *.something and add local7.none).
> 
> Don't forget to tell syslog about the new configuration.
> 
> Hope this helps some.
> -- 
> Robert Segall
> Apsis GmbH
> Postfach, Uetikon am See, CH-8707
> Tel: +41-44-920 4904
> 


Thanks for the tip Robert but that doesn't work, on redhat distros
local7 is already in use for boot.log

I tried --with-log=LOG_POUND but that throws

pound.c:397: `LOG_POUND' undeclared (first use in this function)

On Mon, 19 Sep 2005 13:47:34 +0100 Will Tatam <wtatam(at)premierit.com>
wrote:

> Thanks for the tip Robert but that doesn't work, on redhat distros
> local7 is already in use for boot.log
> 
> I tried --with-log=LOG_POUND but that throws
> 
> pound.c:397: `LOG_POUND' undeclared (first use in this function)
> 

Look in /usr/include/sys/syslog.h for available facilities. You can't
just make one up - you need to use an existing facility such as
LOG_DAEMON, LOG_LOCAL2 or whatever you want. Normally LOG_LOCAL0 to
LOG_LOCAL7 are reserved for private applications, so at least one of
them should be free.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

This is not really a pound problem - set "open files" to a higher value, e.g. 
I''ve set to 8000.
 ulimit -n 8000
 (to see the current value)
ulimit -a
 /David

 On 9/20/05, Alexander Meis <am(at)simoon.de> wrote: 
> 
> Hi...
> 
> i got the Problem that under high traffic sometimes there is:
> Sep 20 10:07:00 cook8 pound: HTTP accept: Too many open files
> Sep 20 10:07:00 cook8 pound: backend xxx.xxx.xxx.xx:80 create: Too many
> open files
> 
> in the log files. So the Server sends 503 Errors.
> 
> Is there some way to fix the Problem?
> 
> Thanks
> 
> Alex
> 
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> 
>
http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-09/1127208579000
>

On Tue, 20 Sep 2005 12:50:31 +0300 david walters <dwalters1(at)gmail.com>
wrote:

> This is not really a pound problem - set "open files" to a higher
> value, e.g. I''ve set to 8000.
>  ulimit -n 8000
>  (to see the current value)
> ulimit -a
>  /David

8000 is probably overkill, though it doesn't really hurt. Most systems
come with a limit of 1K on the possible number of threads, and you use
two descriptors per thread, so 2K (+ a few extra) descriptors would
really be enough.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Maybe it's a pound problem. We started to get the same error under heavy 
load
after the update to the latest pound version (not happend before).
Also the ulimit -n will increase the max number of open file for the current 
shell
session. But what if pound is started from the boot scripts? Maybe adding 
ulimit -n to the startup script will help. Still i think there is some file 
not closed problem
with the latest pound.

On 9/20/05, david walters <dwalters1(at)gmail.com> wrote:
> 
> This is not really a pound problem - set "open files" to a higher value, 
> e.g. I''ve set to 8000.
>  ulimit -n 8000
>  (to see the current value)
> ulimit -a
>  /David
> 
>  On 9/20/05, Alexander Meis <am(at)simoon.de> wrote: 
> > 
> > Hi...
> > 
> > i got the Problem that under high traffic sometimes there is:
> > Sep 20 10:07:00 cook8 pound: HTTP accept: Too many open files 
> > Sep 20 10:07:00 cook8 pound: backend xxx.xxx.xxx.xx:80 create: Too many
> > open files
> > 
> > in the log files. So the Server sends 503 Errors.
> > 
> > Is there some way to fix the Problem?
> > 
>

david walters wrote:
> This is not really a pound problem - set "open files" to a higher value, 
> e.g. I''ve set to 8000.
>  
> ulimit -n 8000
>  
> (to see the current value)
> ulimit -a
>  
> /David

Thanks for helping, i set them to 2048.

Alex

On Wed, 21 Sep 2005 11:15:52 +0900 Stoyan Zhekov <zhekov(at)gmail.com>
wrote:

> Maybe it's a pound problem. We started to get the same error under
> heavy load
> after the update to the latest pound version (not happend before).

Which "latest" do you mean? 1.9 to 1.9.1? 1.8 to 1.9?

> Also the ulimit -n will increase the max number of open file for the
> current shell session. But what if pound is started from the boot
> scripts? Maybe adding ulimit -n to the startup script will help.

It certainly will - the Pound process inherits whatever limits the
parent had - so the shell running the script or the shell you use
interactively are the same.

> Still i think there
> is some file not closed problem
> with the latest pound.

More evidence and details, please.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

> -----Original Message-----
> From: Robert Segall [mailto:roseg(at)apsis.ch]
> Sent: Wednesday, September 21, 2005 10:25 AM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] Open Files
> 
> On Wed, 21 Sep 2005 11:15:52 +0900 Stoyan Zhekov <zhekov(at)gmail.com>
> wrote:
> 
> > Also the ulimit -n will increase the max number of open file for the
> > current shell session. But what if pound is started from the boot
> > scripts? Maybe adding ulimit -n to the startup script will help.
> 
> It certainly will - the Pound process inherits whatever limits the
> parent had - so the shell running the script or the shell you use
> interactively are the same.
> 

You can also use the softlimit program, from the daemontools package by
DJB.  (http://cr.yp.to/daemontools.html)

I run all my pound instances through supervise, with softlimit to limit
open files, data segment size, processes and core files, and with
libsafe.so.2 in my LD_PRELOAD.  Pound runs as a foreground process so
supervise can restart it if it dies.

Works well for me!
Joe Gooch
K12 Systems, Inc.

Dear Joe,

Today i try all patch from you ..,
and there are no error when patching and configure
But when i try to 'make' i got the error like this :

$make
cc -DF_CONF=\"/usr/local/etc/pound.cfg\" -DLINUXCAPS -g -O2 -pthread 
-DAEMON -DUPER -DNEED_STACK  -D_REENTRANT -D_THREAD_SAFE 
-Wstrict-prototypes -pipe   -c -o pound.o pound.c
pound.c:224:28: sys/capability.h: No such file or directory
pound.c: In function `main':
pound.c:885: error: `cap_t' undeclared (first use in this function)
pound.c:885: error: (Each undeclared identifier is reported only once
pound.c:885: error: for each function it appears in.)
pound.c:885: error: syntax error before "caps"
pound.c:886: error: `caps' undeclared (first use in this function)
make: *** [pound.o] Error 1

---
sys/capability.h <--- ???
Currently i install the pound under Mandriva 10.2 ( 2.6.11-6mdk ),

Do i have install SysVinit-2.78-19Au.i386.rpm On my box?

Erwien.


Joe Gooch wrote:
> As I was making the jump from v1.6 to v1.9.1, and having to redo all of
> my patches against pound, I figured it better for me to post these for
> the benefit of others, and additionally so that some (hopefully) could
> be incorporated into the mainstream so I wouldn't have to keep my own
> changes.
> 
> So, here they are, use as you wish, YMMV, yadda yadda.  
> 
> I didn't muck around in the configure script at all, so some of the
> patches will kludge this by adding libs to Makefile.in, or defines there
> as well.  If included in the main codebase those would obviously need to
> be fixed by someone who can read configure files; it just hurts my head
> too much.
> 
> The patches are at:
> https://mgmt.k12system.com/~mrwizard/pound/
> 
> They need to be applied in order, since that's how I built them.  Here
> they are in order:
> 
> pound-1.9.1-nodaemon.patch.gz
> 
> 	Provides a configuration directive called NoDaemon, which makes
> the daemon stay in the foreground instead of forking.  Also, added a
> setsid() call, so regardless, it disassociates from the parent process.
> (might not be portable)
> 
> 	I find it useful, especially when I want to run a pound instance
> (with one config) as a daemon, and another via supervise (daemontools)
> in the foreground.
> 
> pound-1.9.1-linuxcaps.patch.gz
> 
> 	Trivial patch to pound.c, so that after setuid()'s and
> setgid()'s happen, the capabilities set of the process is explicitly set
> to empty.  This is the default for non-root users, but if you've enabled
> a true POSIX capability model (re: http://killa.net/infosec/caps/)  then
> pound will end up with a full inheritable set, which means if, by some
> magic of setuid executables, or something, that the uid gets back to 0,
> full capabilities will be reinstated.
> 
> 	By dropping the inheritable set with a properly modified init or
> kernel, even if the UID somehow returns to 0, there will be no process
> capabilities.  (well, actually those in cap-bound, which in my case, is
> nothing)
> 
> 	It shouldn't break anything, it's in a define (which is on by
> default and not configure'd) and it's likely Linux specific. 
> 
> pound-1.9.1-sessioninfo_statistics_and_logging.patch.gz
> 
> 	This one adds a couple items to Group and Session structures.
> It'll track how many requests hit each backend, each group, etc.  It'll
> track hits/misses against the session database as well, so you can track
> new requests vs. returning requests.  It also allows you to assign a
> GroupName to each group, and institutes loglevel 5 which populates the
> user entry of the CLF and adds the groupname and backend information in
> two preceeding fields.  I think it also lays the groundwork for the
> later authtypes patch, but degrades gracefully.
> 
> 	Starting to get a little more radical.... :)
> 
> pound-1.9.1-authtypes.patch.gz
> 
> 	Adds support for different AuthTypes so that when a request
> comes in that is authenticated, it can track that user in the session
> state and in the logfiles.  I've implemented HTTP Basic Auth and
> Coldfusion <cflogin> style authorization, so that log files can track
> user names of sessions passing through pound.
> 
> pound-1.9.1-redirects.patch.gz
> 
> 	This allows redirects to be properly rewritten even if no Host:
> directive is passed on the request line.  (in which case it will use the
> IP)  It also handles pound running on non-standard ports... It'll
> redirect correctly.  Works well with RewriteRedirect 2, but does include
> functionality not included in RewriteRedirect 2.
> 
> 	Should apply without any of the others.
> 
> pound-1.9.1-listenstatus.patch.gz
> 
> 	Most radical patch.  This allows you to define status sockets,
> in addition to http and https sockets.  You can define a HTTP or Text
> socket.  The text socket will dump out the existing groups, sessions,
> and backends with status information, alive information, etc.  It really
> helps to be able to see what sessions are active now, and whether or not
> a backend is currently flagged as dead or alive.
> 
> 	The HTTP one looks a lot prettier.  It'll wait for a client
> request but it throws the data in the bitbucket and returns a static
> page.  If you don't like my CSS, change it. :-P
> 
> 
> 
> Eventually I'd like to be able to send commands to pound... My current
> brainstorm is the ability to disable a backend from new sessions, but
> allow existing sessions to complete.  (Until they timeout)  That way, if
> I have an issue with a server where I know I'm going to need a reboot,
> and conditions will be degraded until I can, I can just disable new
> connections and when they all expire, reboot the server.  Then, I can
> set the backend to receive connections again.
> 
> It's a long term idea. Not for now.  Peruse, discuss, give me feedback,
> and I'd love to see some (or all) of this appear in the mainstream code.
> 
> Thanks!
> Joe Gooch
> K12 Systems, Inc.
> 
> 
> 

Which "latest" do you mean? 1.9 to 1.9.1? 1.8 to 1.9?


Gentoo 2005.0, pound-1.9.1

More evidence and details, please.


I put a fragment from the log file on:
[ http://rafb.net/paste/results/DXlAfX42.html ]
For bad luck the LogLevel was 0, so not enough information for the requests
:/
Recently I started to think that maybe it's not the pound problem, but we
was just
DDoS-ed/overloaded. Still maybe the log will be interesting for somebody
like
an example of pound under heavy load.

You'll need the libcap utilities.

urpmi libcap1 libcap1-devel libcap-utils


Joe

> -----Original Message-----
> From: Erwien Samantha Y [mailto:erwiensamantha(at)sederhana.or.id]
> Sent: Wednesday, September 21, 2005 11:44 PM
> To: pound(at)apsis.ch
> Subject: Re: [Pound Mailing List] My Patches
> 
> 
> Dear Joe,
> 
> Today i try all patch from you ..,
> and there are no error when patching and configure
> But when i try to 'make' i got the error like this :
> 
> $make
> cc -DF_CONF=\"/usr/local/etc/pound.cfg\" -DLINUXCAPS -g -O2 -pthread
> -DAEMON -DUPER -DNEED_STACK  -D_REENTRANT -D_THREAD_SAFE
> -Wstrict-prototypes -pipe   -c -o pound.o pound.c
> pound.c:224:28: sys/capability.h: No such file or directory
> pound.c: In function `main':
> pound.c:885: error: `cap_t' undeclared (first use in this function)
> pound.c:885: error: (Each undeclared identifier is reported only once
> pound.c:885: error: for each function it appears in.)
> pound.c:885: error: syntax error before "caps"
> pound.c:886: error: `caps' undeclared (first use in this function)
> make: *** [pound.o] Error 1
> 
> ---
> sys/capability.h <--- ???
> Currently i install the pound under Mandriva 10.2 ( 2.6.11-6mdk ),
> 
> Do i have install SysVinit-2.78-19Au.i386.rpm On my box?
> 
> Erwien.
> 
> 
> Joe Gooch wrote:
> > As I was making the jump from v1.6 to v1.9.1, and having to redo all
of
> > my patches against pound, I figured it better for me to post these
for
> > the benefit of others, and additionally so that some (hopefully)
could
> > be incorporated into the mainstream so I wouldn't have to keep my
own
> > changes.
> >
> > So, here they are, use as you wish, YMMV, yadda yadda.
> >
> > I didn't muck around in the configure script at all, so some of the
> > patches will kludge this by adding libs to Makefile.in, or defines
there
> > as well.  If included in the main codebase those would obviously
need to
> > be fixed by someone who can read configure files; it just hurts my
head
> > too much.
> >
> > The patches are at:
> > https://mgmt.k12system.com/~mrwizard/pound/
> >
> > They need to be applied in order, since that's how I built them.
Here
> > they are in order:
> >
> > pound-1.9.1-nodaemon.patch.gz
> >
> > 	Provides a configuration directive called NoDaemon, which makes
> > the daemon stay in the foreground instead of forking.  Also, added a
> > setsid() call, so regardless, it disassociates from the parent
process.
> > (might not be portable)
> >
> > 	I find it useful, especially when I want to run a pound instance
> > (with one config) as a daemon, and another via supervise
(daemontools)
> > in the foreground.
> >
> > pound-1.9.1-linuxcaps.patch.gz
> >
> > 	Trivial patch to pound.c, so that after setuid()'s and
> > setgid()'s happen, the capabilities set of the process is explicitly
set
> > to empty.  This is the default for non-root users, but if you've
enabled
> > a true POSIX capability model (re: http://killa.net/infosec/caps/)
then
> > pound will end up with a full inheritable set, which means if, by
some
> > magic of setuid executables, or something, that the uid gets back to
0,
> > full capabilities will be reinstated.
> >
> > 	By dropping the inheritable set with a properly modified init or
> > kernel, even if the UID somehow returns to 0, there will be no
process
> > capabilities.  (well, actually those in cap-bound, which in my case,
is
> > nothing)
> >
> > 	It shouldn't break anything, it's in a define (which is on by
> > default and not configure'd) and it's likely Linux specific.
> >
> > pound-1.9.1-sessioninfo_statistics_and_logging.patch.gz
> >
> > 	This one adds a couple items to Group and Session structures.
> > It'll track how many requests hit each backend, each group, etc.
It'll
> > track hits/misses against the session database as well, so you can
track
> > new requests vs. returning requests.  It also allows you to assign a
> > GroupName to each group, and institutes loglevel 5 which populates
the
> > user entry of the CLF and adds the groupname and backend information
in
> > two preceeding fields.  I think it also lays the groundwork for the
> > later authtypes patch, but degrades gracefully.
> >
> > 	Starting to get a little more radical.... :)
> >
> > pound-1.9.1-authtypes.patch.gz
> >
> > 	Adds support for different AuthTypes so that when a request
> > comes in that is authenticated, it can track that user in the
session
> > state and in the logfiles.  I've implemented HTTP Basic Auth and
> > Coldfusion <cflogin> style authorization, so that log files can
track
> > user names of sessions passing through pound.
> >
> > pound-1.9.1-redirects.patch.gz
> >
> > 	This allows redirects to be properly rewritten even if no Host:
> > directive is passed on the request line.  (in which case it will use
the
> > IP)  It also handles pound running on non-standard ports... It'll
> > redirect correctly.  Works well with RewriteRedirect 2, but does
include
> > functionality not included in RewriteRedirect 2.
> >
> > 	Should apply without any of the others.
> >
> > pound-1.9.1-listenstatus.patch.gz
> >
> > 	Most radical patch.  This allows you to define status sockets,
> > in addition to http and https sockets.  You can define a HTTP or
Text
> > socket.  The text socket will dump out the existing groups,
sessions,
> > and backends with status information, alive information, etc.  It
really
> > helps to be able to see what sessions are active now, and whether or
not
> > a backend is currently flagged as dead or alive.
> >
> > 	The HTTP one looks a lot prettier.  It'll wait for a client
> > request but it throws the data in the bitbucket and returns a static
> > page.  If you don't like my CSS, change it. :-P
> >
> >
> >
> > Eventually I'd like to be able to send commands to pound... My
current
> > brainstorm is the ability to disable a backend from new sessions,
but
> > allow existing sessions to complete.  (Until they timeout)  That
way, if
> > I have an issue with a server where I know I'm going to need a
reboot,
> > and conditions will be degraded until I can, I can just disable new
> > connections and when they all expire, reboot the server.  Then, I
can
> > set the backend to receive connections again.
> >
> > It's a long term idea. Not for now.  Peruse, discuss, give me
feedback,
> > and I'd love to see some (or all) of this appear in the mainstream
code.
> >
> > Thanks!
> > Joe Gooch
> > K12 Systems, Inc.
> >
> >
> >
> 
> 
> --
> To unsubscribe send an email with subject 'unsubscribe' to
pound(at)apsis.ch.
> Please contact roseg(at)apsis.ch for questions.
> http://192.168.1.2:8080/Apsis/pound/pound_list/archive/2005/2005-
> 09/1125580698000/1127360661000

On Thu, 22 Sep 2005 14:09:35 +0900 Stoyan Zhekov <zhekov(at)gmail.com>
wrote:

> Which "latest" do you mean? 1.9 to 1.9.1? 1.8 to 1.9?
> 
> 
> Gentoo 2005.0, pound-1.9.1
> 
> More evidence and details, please.
> 
> 
> I put a fragment from the log file on:
> [ http://rafb.net/paste/results/DXlAfX42.html ]
> For bad luck the LogLevel was 0, so not enough information for the
> requests:/
> Recently I started to think that maybe it's not the pound problem, but
> we was just
> DDoS-ed/overloaded. Still maybe the log will be interesting for
> somebody like
> an example of pound under heavy load.

Certainly interesting. You definitely have an overloaded system - but it
degrades nicely: your back-ends get overloaded before Pound. Please
notice you start having back-end time-outs, followed by "out of file
descriptors" (as you have more and more connections waiting you start
running out of them).

You may want to try:

- increasing the file-descriptor limit, as described in another post.
- adding more back-ends, so that each of them is faster to respond.
- increasing the Server time-out, so that Pound waits longer for a
response.
- decreasing the Client so that at least some connections get killed
early.

Have fun.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904