We just got this in our logs this morning and Pound stopped processing 
requests..

Apr 15 06:22:34 fantomen pound: HTTP pthread_create: Cannot allocate memory
Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
Apr 15 08:42:35 fantomen pound: HTTP pthread_create: Cannot allocate memory
Apr 15 09:24:26 fantomen pound: HTTP pthread_create: Cannot allocate memory

With just 256MBs of RAM the rig itself is a bit underpowered i know, but 
still it had 500 Megs of Swap to use.
Restarting Pound fixed it but im curious of what caused this, we have 
the lowest server usage in this timeframe.
Running Red Hat 9 and Pound 1.6.

Best Regards
Joakim Thornberg

On Thu, 2004-04-15 at 10:10, Joakim Thornberg wrote:
> We just got this in our logs this morning and Pound stopped processing 
> requests..
> 
> Apr 15 06:22:34 fantomen pound: HTTP pthread_create: Cannot allocate memory
> Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
> Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
> Apr 15 08:42:35 fantomen pound: HTTP pthread_create: Cannot allocate memory
> Apr 15 09:24:26 fantomen pound: HTTP pthread_create: Cannot allocate memory
> 
> With just 256MBs of RAM the rig itself is a bit underpowered i know, but 
> still it had 500 Megs of Swap to use.
> Restarting Pound fixed it but im curious of what caused this, we have 
> the lowest server usage in this timeframe.
> Running Red Hat 9 and Pound 1.6.

256MB of RAM should be more than enough. I suspect you run into one of
two possible issues:

- you may have reached the allowed limit on the number of processes
(system-configuration and shell dependent) or threads (normally just
over 1000, can be changed by recompiling libpthread). Either may occur
under heavy load and/or long-lived requests (or some kind of intentional
DoS).

- given that this is RH9: this may just be a problem of the NPTL, which
has bit quite a few people in the past.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-1-920 4904

Robert Segall wrote:

>On Thu, 2004-04-15 at 10:10, Joakim Thornberg wrote:
>  
>
>>We just got this in our logs this morning and Pound stopped processing 
>>requests..
>>
>>Apr 15 06:22:34 fantomen pound: HTTP pthread_create: Cannot allocate memory
>>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
>>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate memory
>>Apr 15 08:42:35 fantomen pound: HTTP pthread_create: Cannot allocate memory
>>Apr 15 09:24:26 fantomen pound: HTTP pthread_create: Cannot allocate memory
>>
>>With just 256MBs of RAM the rig itself is a bit underpowered i know, but 
>>still it had 500 Megs of Swap to use.
>>Restarting Pound fixed it but im curious of what caused this, we have 
>>the lowest server usage in this timeframe.
>>Running Red Hat 9 and Pound 1.6.
>>    
>>
>
>256MB of RAM should be more than enough. I suspect you run into one of
>two possible issues:
>
>- you may have reached the allowed limit on the number of processes
>(system-configuration and shell dependent) or threads (normally just
>over 1000, can be changed by recompiling libpthread). Either may occur
>under heavy load and/or long-lived requests (or some kind of intentional
>DoS).
>
>- given that this is RH9: this may just be a problem of the NPTL, which
>has bit quite a few people in the past.
>  
>
Thanks for your reply Robert.
Considering RH9 is dropping from Red Hats supported list i guess we have 
to look for another Dist or OS to replace it.
What is considered the best Pound platform ? We've tried FreeBSD but 
pound simply would not compile.
I've seen people having problems with RedHat Enterprise aswell as SuSe 9.0.

Also is this number of processes setting something that can be set at 
install ?

Best Regards
Joakim Thornberg

	I've run into the same quandary now myself.  In the process I have
decent luck with Fedora Core1 & Core2 Test2 for Linux, and I run OpenBSD 3.4
for my main production Pound.  Now OpenBSD does not let Pound compile out of
the box, but it isn't bad.  Here are the short set of steps, if there is
interest I will do a full write up.

	1)Grab OpenSSL and compile and install with thread support in an
alternative location.
	2)Grab Daemontools (my personal preference in startup) and install
it.
	3)Configure Pound to point to the alternative SSL, and disable
daemon support.
	4)Make and install pound.
	5)Add a Pound entry and config file for each instance you want to
the services.

	By using Daemontools you will get an installation that will restart
in the case of an occasional failure.  We run a lot of web traffic through
this on a 24x7 production site and have yet to have lost uptime.

	Good Luck,
		Shawn



> -----Original Message-----
> From: Joakim [mailto:joakim(at)regerar.nu]
> Sent: Friday, April 16, 2004 6:21 AM
> To: pound(at)apsis.ch
> Subject: Re: HTTP pthread_create: Cannot allocate memory
> 
> Robert Segall wrote:
> 
> >On Thu, 2004-04-15 at 10:10, Joakim Thornberg wrote:
> >
> >
> >>We just got this in our logs this morning and Pound stopped processing
> >>requests..
> >>
> >>Apr 15 06:22:34 fantomen pound: HTTP pthread_create: Cannot allocate
> memory
> >>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate
> memory
> >>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate
> memory
> >>Apr 15 08:42:35 fantomen pound: HTTP pthread_create: Cannot allocate
> memory
> >>Apr 15 09:24:26 fantomen pound: HTTP pthread_create: Cannot allocate
> memory
> >>
> >>With just 256MBs of RAM the rig itself is a bit underpowered i know, but
> >>still it had 500 Megs of Swap to use.
> >>Restarting Pound fixed it but im curious of what caused this, we have
> >>the lowest server usage in this timeframe.
> >>Running Red Hat 9 and Pound 1.6.
> >>
> >>
> >
> >256MB of RAM should be more than enough. I suspect you run into one of
> >two possible issues:
> >
> >- you may have reached the allowed limit on the number of processes
> >(system-configuration and shell dependent) or threads (normally just
> >over 1000, can be changed by recompiling libpthread). Either may occur
> >under heavy load and/or long-lived requests (or some kind of intentional
> >DoS).
> >
> >- given that this is RH9: this may just be a problem of the NPTL, which
> >has bit quite a few people in the past.
> >
> >
> Thanks for your reply Robert.
> Considering RH9 is dropping from Red Hats supported list i guess we have
> to look for another Dist or OS to replace it.
> What is considered the best Pound platform ? We've tried FreeBSD but
> pound simply would not compile.
> I've seen people having problems with RedHat Enterprise aswell as SuSe
> 9.0.
> 
> Also is this number of processes setting something that can be set at
> install ?
> 
> Best Regards
> Joakim Thornberg

I've just abandoned RH9 as well at its support EOL.  I've switched over to
Gentoo -- so far, it's been working great, and Pound seems to be very stable.
One thing missing is an ebuild for Pound-1.7, but it is EXTREMELY easy to make
your own until they get one into the official portage tree (I use the portage
overlay feature for my own custom ebuild).  Let me know if you'd like any
assistance with this, as I just got it all nailed down myself.

Oh, I should also mention that I'm running on the 2.6 kernel -- seems to
be working fine for me...  (note that I don't run an extremely high
traffic site though -- I just have lots of various backends)

-jason

On Fri, Apr 16, 2004 at 12:21:14PM +0200, Joakim wrote:
> Robert Segall wrote:
> 
> >On Thu, 2004-04-15 at 10:10, Joakim Thornberg wrote:
> > 
> >
> >>We just got this in our logs this morning and Pound stopped processing 
> >>requests..
> >>
> >>Apr 15 06:22:34 fantomen pound: HTTP pthread_create: Cannot allocate 
> >>memory
> >>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate 
> >>memory
> >>Apr 15 08:42:35 fantomen pound: HTTPS pthread_create: Cannot allocate 
> >>memory
> >>Apr 15 08:42:35 fantomen pound: HTTP pthread_create: Cannot allocate 
> >>memory
> >>Apr 15 09:24:26 fantomen pound: HTTP pthread_create: Cannot allocate 
> >>memory
> >>
> >>With just 256MBs of RAM the rig itself is a bit underpowered i know, but 
> >>still it had 500 Megs of Swap to use.
> >>Restarting Pound fixed it but im curious of what caused this, we have 
> >>the lowest server usage in this timeframe.
> >>Running Red Hat 9 and Pound 1.6.
> >>   
> >>
> >
> >256MB of RAM should be more than enough. I suspect you run into one of
> >two possible issues:
> >
> >- you may have reached the allowed limit on the number of processes
> >(system-configuration and shell dependent) or threads (normally just
> >over 1000, can be changed by recompiling libpthread). Either may occur
> >under heavy load and/or long-lived requests (or some kind of intentional
> >DoS).
> >
> >- given that this is RH9: this may just be a problem of the NPTL, which
> >has bit quite a few people in the past.
> > 
> >
> Thanks for your reply Robert.
> Considering RH9 is dropping from Red Hats supported list i guess we have 
> to look for another Dist or OS to replace it.
> What is considered the best Pound platform ? We've tried FreeBSD but 
> pound simply would not compile.
> I've seen people having problems with RedHat Enterprise aswell as SuSe 9.0.
> 
> Also is this number of processes setting something that can be set at 
> install ?
> 
> Best Regards
> Joakim Thornberg
> 

On Fri, 2004-04-16 at 12:21, Joakim wrote:
> Thanks for your reply Robert.
> Considering RH9 is dropping from Red Hats supported list i guess we have 
> to look for another Dist or OS to replace it.
> What is considered the best Pound platform ? We've tried FreeBSD but 
> pound simply would not compile.
> I've seen people having problems with RedHat Enterprise aswell as SuSe 9.0.

FreeBSD (or any other *BSD) works fine - we have been using OpenBSD for
a long time now. The main issue is that *BSD comes with OpenSSL without
thread support, thus you have to compile and install your own version.
Once you do that Pound compiles and runs just fine. We normally run it
with daemontools, but that is a matter of taste.

On the Linux side we've had excellent results with Gentoo and Suse
(though admittedly an older version), both of which we use internally.
We have had reports for lots of others as well - Debian, older RH, newer
RH (EVx), etc.

> Also is this number of processes setting something that can be set at 
> install ?

Nothing to do with Pound: this is a limitation in the threads library
(change a header and recompile libpthread if you need more than 1K
concurrent threads) and in the system (see the Pound web page for
details on how to change your limits at runtime - these are mostly shell
variables).
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-1-920 4904