For your collective interest and comment...

Regarding my recent questions about three way pound setup, with server 
affinity, I've come up with a solution that should work. In theory...

Reminder: I'm having one "front" pound, which distributes requests 
between two "back" pound servers, which in turn send requests to 
back-end servers. We want client stiockiness with the back end server 
(on a per session basis).
There's an ascii diagram of my current idea for the setup here:

http://ohmslaw.org.uk/threeWayPoundSetup.txt

(propertional font needed!)

The key thing is that both the "front" pound and the "back" pound 
servers have sesssion stickyness, and there's a certian timeout 
equality: the tomcat session timeout is slightly less than the back end 
pound proxies timeout, which is turn is slightly less than the front 
pound proxy timeout....


Also, we've chatted about the idea of not requiring session stickyness, 
as has been suggested, and we like the idea. May make it happen with 
some of our products, just not quite yet!

Alex

Hi

If I understand your setup and your intentions correctly I am not sure if 
this will work. Quite possibly I have missed something though...

I assume you use Session Type COOKIE with ID JSESSIONID, and that all 
config that might be changed go on the back pound balancers.

A request enters the front pound balancer, and this randomly chooses one 
of the back pound balancers to process the request. The back balancer then 
randomly chooses a backend server to process the request. When the backend 
responds the chosen back pound balancer creates a session entry for the 
JSESSIONID that the backend tries to set, associating this JSESSIONID with 
the chosen backend. When the front pound balancer sees the response it 
does the same, associating the chosen back pound balancer with the 
JSESSIONID.

Now if you are to reconfigure pound then you first change the config for 
back1, stop it and start it up again. At this point 2 bad things will 
happen:

1: If you do not use the Grace feature all open tcp sessions through back1
    will break, resulting in cancelled downloads, broken images and such.
    Using the Grace feature will prevent this.

2: The clients that used to be directed through back1 by the front pound
    balancer will now be sent through back2. Since back2 doesn't know
    anything about these JSESSIONID's a new backend will be selected at
    random. Most likely the client will end up on a different backend than
    before, and his session will break.

Of course, things get even worse when you have started back1 up again and 
you are restarting back2. All sessions are now going throug back2, which 
you are stopping, and are moved to back1 by the front pound balancer. 
back1 knows nothing about any sessions, so random backends are selected 
for all users.

Of course users that has left their keyboard and mouse alone during the 
entire restarting process will end up at the same back pound balancer the 
next time they load a page, but their sessions in back1 and back2 are 
gone, so they also end up at random backend servers.

If you are using just one pound instance with the Grace feature you will 
reject connections for a couple of milliseconds or so between sending the 
running pound instance the HUP signal and starting the new one. And no 
user will have their sessions on the backends ruined twice.

Of course you can write a script that uses poundctl to periodically 
synchronize the session table between the two back pound balancers. But I 
think it will be simpler to write a restart script that dumps the session 
table of the old pound instance and uploads it to the newly started one. 
Some clients will be allocated random backends before your script has 
uploaded the session table again. Probably uploading the sessions won't 
take much time, so it shouldn't be a big problem.


Regards
Rune

---
Rune Sætre <rune.saetre(at)netcom-gsm.no>
NetCom as
..

On Fri, 14 Sep 2007, Alex Hunsley wrote:
[...]

Hi Rune
Thanks for your reply, I really appreciate you giving this thought!
[...]

Actually, due to a technical limitation, we have to use URL rewriting, so
JSESSIONID appears as part of the URL. I was going to add my voice to that
other poster, btw, in saying "can we please have rewritten URLs as a session
type?" So we can't use JSESSIONID cookie, so it was going to be IP.
(We can't use cookies in general. This is a limitation of mobile app
programming, sadly - not all phones behave well with HTTP (via Java)
cookies....)

Now, it seems session type of IP isn't deterministic - i.e. a pound proxy, on
seeing a certain IP for first time, will choose one of N backends at random -
it won't base decision on a hash of the IP (I tested this) - which is a shame,
since that would make my life a lot easier. I wonder if there is any chance of
having deterministic IP hash as an option for the IP session type?

What is the best avenue for requesting changes/enhancements to Pound, by the
way? This mailing list?
[...]


Sorry Rune, my omission, in saying that to reconfigure a proxy I would:

1) use poundctl to tell that proxy to stop taking new connections
2) wait until no sessions are active in that proxy
3) reeconfig that proxy, restart
4) use poundctl to tell that proxy to start taking new connections

[...]

I was wondering about using the grace feature, but I don't think we can use it,
since our application currently requires sticky sessions with a back-end proxy.
In other words, it's not enough to ensure that current downloads are services
before a pound instance dies - we need to ensure a certain client always talks
to the same proxy server for the lifetime of their application session, which
consists of many requests...

[...]

Is that possible? How? I can't see any info on how to use poundctl to dump
session info. I know you can remove and add session keys....
[...]


thanks for your thoughts!
Alex

Hi

I finally get it, you are using poundctl on the front balancer to disable 
new sessions to one of the back servers until it is out of sessions. I 
believe that should work, yes.

When running poundctl without arguments (except for the location of the 
control socket) it dumps the configuration and the session table to 
stdout.

When firing up Pound-2.3.2 here now I notice that it takes several seconds 
from it is started to it starts accepting connections. Is this normal?
It kind of foils my cunning plan for a quick restart using just the Grace 
feature.

Rune

---
Rune Sætre <rune.saetre(at)netcom-gsm.no>
NetCom as
..

On Mon, 17 Sep 2007, Alex Hunsley wrote:
[...][...][...][...][...][...][...][...][...][...][...]

Rune Saetre wrote:[...]
Hi Rune
Yes, that's it. Sorry for any confusion by me not making that clear![...]
Ah, I know about this part of poundctl, I just assumed it wasn't in a 
format that allowed the session info to be reproduced/reimported later....[...]
Will let you know if I see that behaviour....
Alex
[...]