/ Zope / Apsis / Pound Mailing List / Archive / 2007 / 2007-12 / Failed requests after disabling a backend

[ << ] [ >> ]

[ Internal server error for long requests / Pete ... ] [ Can't Get Pound Up / "Victor Subervi" ... ]

Failed requests after disabling a backend
Steve <spm(at)fostam.franken.de>
2007-12-18 22:23:08 [ FULL ]
I've encountered a problem when disabling a backend with poundctl in a
non-session-based setup:
After disabling the backend, for some time (i.e. up to the value of
"Alive") some requests create a "no back-end" error.

This seems to be due to the fact that when a backend is disabled through
poundctl, the total priority for its service (svc->tot_pri) is not
updated immediately, but only on the next run of "do_resurect()", which
is called only every "Alive" seconds.

As an effect, sometimes the while loop in "rand_backend()" will never
reach a pri<0 although suitable backends would be available, because the
random priority it was called with was based on a too high svc->tot_pri
value. The higher the disabled backend's priority had been, the more
requests will falsely fail, creating lots of unhappy users.

[The same effect occurs the other way round (when enabling the backend,
it takes up to "Alive" seconds until it's used), although the problem is
uncritical there because users don't get errors, the use of the new
backend is just delayed for some seconds.]

So my question is:  would it be possible (i.e. thread-safe) to
immediately update the service's priority like this (svc.c):

        case CTRL_DE_BE:
           ...
           be->disabled = 1;
           svc->tot_pri -= be->priority;

I suspect this could lead to problems, e.g. if the timer thread is in
the middle of "do_resurect()" when the above section is called.
However, there is a function "disable_be()" which recalculates
svc->tot_pri and also does thread locking, but it doesn't seem to be
used at all...?

Please clarify...  thanks!

Steve

Re: [Pound Mailing List] Failed requests after disabling a backend
Robert Segall <roseg(at)apsis.ch>
2007-12-24 13:39:21 [ FULL ]
On Tue, 2007-12-18 at 22:23 +0100, Steve wrote:[...]

Thanks Steve - we'll look into this and fix it for the next release.[...]

MailBoxer