Hi Folks,

I wonder if anyone can point me in the right direction.

I have an OpenBSD 3.9 (Release) box on which I have successfully 
installed OpenSSL 9.8 and Pound 2.1 and Pound 1.10. Behind the Pound box 
are two IIS6 web servers that I want to load balance / session track 
against.

Ok so the problem I have is this when running v1.10 with its own config 
files etc everything works a treat, client A connects to server1 client 
B connects to server2, client C to server 1 etc.

No when I move to v2.1 Clients A/B/C all connect to Server1, nothing 
ever connects to server2.

When I tried v2.0 previously I saw the same behavior.

Any ideas where I have gone wrong?

Below is my pound 2.1 cfg file

User "nobody"
Group "nobody"
Daemon 1
LogFacility user
LogLevel 2
Alive 30

ListenHTTP
            Address 10.190.10.89
            Port    80

            Service

                BackEnd
                    Address 10.190.10.70    <--- IIS Server1
                    Port    80
                End

                BackEnd
                    Address 10.190.10.71    <--- IIS Server2
                    Port    80
                End

                Session
                    Type  IP
                    TTL  300
                End

            End
End

Below is the corresponding v1 cfg file that works

ListenHTTP      10.190.10.89,80
User            nobody
Group           nobody
LogFacility     user
LogLevel        2
Alive           30
Client          10

UrlGroup        ".*"
BackEnd         10.190.10.70,80,1
BackEnd         10.190.10.71,80,1
Session         IP      300
EndGroup

On Thu, 2006-08-17 at 13:19 +0100, Simon Slaytor wrote:
> Hi Folks,
> 
> I wonder if anyone can point me in the right direction.
> 
> I have an OpenBSD 3.9 (Release) box on which I have successfully 
> installed OpenSSL 9.8 and Pound 2.1 and Pound 1.10. Behind the Pound box 
> are two IIS6 web servers that I want to load balance / session track 
> against.
> 
> Ok so the problem I have is this when running v1.10 with its own config 
> files etc everything works a treat, client A connects to server1 client 
> B connects to server2, client C to server 1 etc.
> 
> No when I move to v2.1 Clients A/B/C all connect to Server1, nothing 
> ever connects to server2.
> 
> When I tried v2.0 previously I saw the same behavior.
> 
> Any ideas where I have gone wrong?

Try removing the Session from your 2.1 config and see if it helps, as it
should go to random distribution. Otherwise the behaviour should be
identical - Session IP uses a hash of the client address to select the
back-end.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

Thanks for that yes, removing the 'Session' tag works, both backend 
servers are now used.

Unfortunately I need to track sessions, the backend servers will be 
running an ASP.NET app, a financial calculation engine. As such a client 
PC will request a session, submit parameters for a quote, retrieve the 
quote results etc, etc. Unfortunately I'm not the programmer of the App 
so cannot dictate they code it to cope with multiple backend requesters etc.

Any idea how to solve this as v1.10 appears to work well. This initial 
deployment is for 'internal' applications, however the plan is to use 
this approach and provide an Internet SOAP interface to the ASP.NET app, 
this link being secured with SSL server certificates. I had planned on 
using Pound as the SSL offload engine as well as a LoadBalancer. All 
doable in v1.10 I know but I'd prefer to keep pace with the latest 
developments.

Failing this is there any reason not to go with the v1.10 release for 
this project?

Many thanks

Simon


Robert Segall wrote:
> On Thu, 2006-08-17 at 13:19 +0100, Simon Slaytor wrote:
>   
>> Hi Folks,
>>
>> I wonder if anyone can point me in the right direction.
>>
>> I have an OpenBSD 3.9 (Release) box on which I have successfully 
>> installed OpenSSL 9.8 and Pound 2.1 and Pound 1.10. Behind the Pound box 
>> are two IIS6 web servers that I want to load balance / session track 
>> against.
>>
>> Ok so the problem I have is this when running v1.10 with its own config 
>> files etc everything works a treat, client A connects to server1 client 
>> B connects to server2, client C to server 1 etc.
>>
>> No when I move to v2.1 Clients A/B/C all connect to Server1, nothing 
>> ever connects to server2.
>>
>> When I tried v2.0 previously I saw the same behavior.
>>
>> Any ideas where I have gone wrong?
>>     
>
> Try removing the Session from your 2.1 config and see if it helps, as it
> should go to random distribution. Otherwise the behaviour should be
> identical - Session IP uses a hash of the client address to select the
> back-end.
>   

On Thu, 2006-08-17 at 15:53 +0100, Simon Slaytor wrote:
> Hi Robert,
> 
> Thanks for that yes, removing the 'Session' tag works, both backend 
> servers are now used.

Good - so basic load-balancing is OK.

> Unfortunately I need to track sessions, the backend servers will be 
> running an ASP.NET app, a financial calculation engine. As such a client 
> PC will request a session, submit parameters for a quote, retrieve the 
> quote results etc, etc. Unfortunately I'm not the programmer of the App 
> so cannot dictate they code it to cope with multiple backend requesters etc.

You don't need to. The suggestion to remove Session was just for
testing, to make sure that Pound recognises both back-ends.

The IP sessions are equivalent to the old "sticky" sessions: an IP
address is always mapped to the same back-end. It may well be that in
your test you used two clients from the same IP address (possibly via
NAT) and thus both were sent to the same back-end. Try the same with
more clients, from various addresses, and you should see them
distributed correctly.

Even better: if at all possible use some other kind of session
mechanism, such as COOKIE or PARM.

> Any idea how to solve this as v1.10 appears to work well. This initial 
> deployment is for 'internal' applications, however the plan is to use 
> this approach and provide an Internet SOAP interface to the ASP.NET app, 
> this link being secured with SSL server certificates. I had planned on 
> using Pound as the SSL offload engine as well as a LoadBalancer. All 
> doable in v1.10 I know but I'd prefer to keep pace with the latest 
> developments.
> 
> Failing this is there any reason not to go with the v1.10 release for 
> this project?

1.x is no longer updated - all fixes go into the 2.x series. 1.10 should
be "more or less" OK, so for the moment it should be usable, but there
are no guarantees for the future.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

Thanks for the reply, sorry I now see what your saying re session's.

I'd hoped to use IP session tracking as again it requires nothing from 
the programmers.

The 'clients' from which I have been testing and those the 'production' 
system will be used with all exist on the same class B network (10.190) 
as the pound box itself so no NAT issues.

Specifically 10.190.66.66 / 10.190.70.70 / 10.190.70.77 from these three 
clients I'm always connected to Server 1 when running v2.1, as 
previously stated using v1 I get evenly distributed. I have just tried 
connecting to v2.1 from the following 'remote routed' hosts 
172.16.10.5/24 and 192.168.1.181/24 all five hosts connect to server1.

Any suggestions what to look at next.

Again many thanks for taking the time to reply.

Regards

Simon

Robert Segall wrote:
> On Thu, 2006-08-17 at 15:53 +0100, Simon Slaytor wrote:
>   
>> Hi Robert,
>>
>> Thanks for that yes, removing the 'Session' tag works, both backend 
>> servers are now used.
>>     
>
> Good - so basic load-balancing is OK.
>
>   
>> Unfortunately I need to track sessions, the backend servers will be 
>> running an ASP.NET app, a financial calculation engine. As such a client 
>> PC will request a session, submit parameters for a quote, retrieve the 
>> quote results etc, etc. Unfortunately I'm not the programmer of the App 
>> so cannot dictate they code it to cope with multiple backend requesters etc.
>>     
>
> You don't need to. The suggestion to remove Session was just for
> testing, to make sure that Pound recognises both back-ends.
>
> The IP sessions are equivalent to the old "sticky" sessions: an IP
> address is always mapped to the same back-end. It may well be that in
> your test you used two clients from the same IP address (possibly via
> NAT) and thus both were sent to the same back-end. Try the same with
> more clients, from various addresses, and you should see them
> distributed correctly.
>
> Even better: if at all possible use some other kind of session
> mechanism, such as COOKIE or PARM.
>
>   
>> Any idea how to solve this as v1.10 appears to work well. This initial 
>> deployment is for 'internal' applications, however the plan is to use 
>> this approach and provide an Internet SOAP interface to the ASP.NET app, 
>> this link being secured with SSL server certificates. I had planned on 
>> using Pound as the SSL offload engine as well as a LoadBalancer. All 
>> doable in v1.10 I know but I'd prefer to keep pace with the latest 
>> developments.
>>
>> Failing this is there any reason not to go with the v1.10 release for 
>> this project?
>>     
>
> 1.x is no longer updated - all fixes go into the 2.x series. 1.10 should
> be "more or less" OK, so for the moment it should be usable, but there
> are no guarantees for the future.
>   

On Thu, 2006-08-17 at 21:05 +0100, Simon Slaytor wrote:
> Hi Robert,
> 
> Thanks for the reply, sorry I now see what your saying re session's.
> 
> I'd hoped to use IP session tracking as again it requires nothing from 
> the programmers.

The other session types also require nothing - they use existing
mechanisms.

> The 'clients' from which I have been testing and those the 'production' 
> system will be used with all exist on the same class B network (10.190) 
> as the pound box itself so no NAT issues.
> 
> Specifically 10.190.66.66 / 10.190.70.70 / 10.190.70.77 from these three 
> clients I'm always connected to Server 1 when running v2.1, as 
> previously stated using v1 I get evenly distributed. I have just tried 
> connecting to v2.1 from the following 'remote routed' hosts 
> 172.16.10.5/24 and 192.168.1.181/24 all five hosts connect to server1.
> 
> Any suggestions what to look at next.
> 
> Again many thanks for taking the time to reply.
> 
> Regards
> 
> Simon

Sounds like a real problem. If you could check on what Pound thinks
about the client address (file svc.c, function get_backend(), print out
the contents of from_host) it would be really helpful to try and see
what exactly goes wrong.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

Ok, I'm no programmer so forgive me if this is wrong.

I've added the following line to the SESS_IP section in svc.c, 
immediately following the pri = 0; line.

printf("Host: %d\n",from_host);

I've also added the following line, immediately before the break; statement.

printf("Backend %d\n",res);

I've then set pound to not run as a daemon.

Ok the the following is the output I get.

When connecting from host 10.190.66.66

Host: 1111670282
Backend -1957071616

When connecting from host 10.190.70.70

Host: 1179041290
Backend -1957071616

As a matter of interest I also added the latter printf statement just 
prior to the break; in the SESS_NONE section.

When running with 'no sessions' I get the following output:

Backend -2029771520   <- client connects to server 1
Backend -2029771008   <-client connects to server 2

The sequence is obviously random.

I hope thats what your after, if not treat me as an idiot and tell me 
what you want.

Many thanks

Simon


Robert Segall wrote:
> On Thu, 2006-08-17 at 21:05 +0100, Simon Slaytor wrote:
>   
>> Hi Robert,
>>
>> Thanks for the reply, sorry I now see what your saying re session's.
>>
>> I'd hoped to use IP session tracking as again it requires nothing from 
>> the programmers.
>>     
>
> The other session types also require nothing - they use existing
> mechanisms.
>
>   
>> The 'clients' from which I have been testing and those the 'production' 
>> system will be used with all exist on the same class B network (10.190) 
>> as the pound box itself so no NAT issues.
>>
>> Specifically 10.190.66.66 / 10.190.70.70 / 10.190.70.77 from these three 
>> clients I'm always connected to Server 1 when running v2.1, as 
>> previously stated using v1 I get evenly distributed. I have just tried 
>> connecting to v2.1 from the following 'remote routed' hosts 
>> 172.16.10.5/24 and 192.168.1.181/24 all five hosts connect to server1.
>>
>> Any suggestions what to look at next.
>>
>> Again many thanks for taking the time to reply.
>>
>> Regards
>>
>> Simon
>>     
>
> Sounds like a real problem. If you could check on what Pound thinks
> about the client address (file svc.c, function get_backend(), print out
> the contents of from_host) it would be really helpful to try and see
> what exactly goes wrong.
>   

On Fri, 2006-08-18 at 11:53 +0100, Simon Slaytor wrote:
> Hi Robert,
> 
> Ok, I'm no programmer so forgive me if this is wrong.
> 
> I've added the following line to the SESS_IP section in svc.c, 
> immediately following the pri = 0; line.
> 
> printf("Host: %d\n",from_host);

from_host is structure. Try printing out addr instead, as it contains
just the information you want. Ideally use inet_ntoa for a
human-readable string.

> I've also added the following line, immediately before the break; statement.
> 
> printf("Backend %d\n",res);

res is a pointer to a structure, but the results are good enough.

> I've then set pound to not run as a daemon.
> 
> Ok the the following is the output I get.
> 
> When connecting from host 10.190.66.66
> 
> Host: 1111670282
> Backend -1957071616
> 
> When connecting from host 10.190.70.70
> 
> Host: 1179041290
> Backend -1957071616
> 
> As a matter of interest I also added the latter printf statement just 
> prior to the break; in the SESS_NONE section.
> 
> When running with 'no sessions' I get the following output:
> 
> Backend -2029771520   <- client connects to server 1
> Backend -2029771008   <-client connects to server 2
> 
> The sequence is obviously random.
> 
> I hope thats what your after, if not treat me as an idiot and tell me 
> what you want.

That is almost what we need. If you could change the printing of
from_host as described above it would be very helpful, if you could also
show the value of pri it would be even better. The code would look a bit
like this:

    case SESS_IP:
        /* "sticky" mappings */
fprintf(stderr, "from_host: %s\n", inet_ntoa(&from_host));
        addr = from_host.s_addr;
        pri = 0;
        while(addr) {
            pri = (pri << 3) ^ (addr & 0xff);
            addr = (addr >> 8);
        }
fprintf(stderr, "pri: %d\n", pri & 0xffff);
        res = rand_backend(svc->backends, (pri & 0xffff) %svc->tot_pri);
fprintf(stderr, "backend %ld\n",res);
        break;

I sincerely hope you don't think this is idiot-like treatment - we are
certainly interested in your feedback and we would like to fix the bug
rather than insult people.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904

Hi Robert,

Thanks again for the time, I will make the changes tomorrow and resubmit 
the results.

Not for one minute have I been made to feel like an idiot, you miss 
understand my turn of phrase, I was just trying to say that I'm no 
programmer so you might need to spell things out a little more for me.

Regards

Simon


Robert Segall wrote:
> On Fri, 2006-08-18 at 11:53 +0100, Simon Slaytor wrote:
>   
>> Hi Robert,
>>
>> Ok, I'm no programmer so forgive me if this is wrong.
>>
>> I've added the following line to the SESS_IP section in svc.c, 
>> immediately following the pri = 0; line.
>>
>> printf("Host: %d\n",from_host);
>>     
>
> from_host is structure. Try printing out addr instead, as it contains
> just the information you want. Ideally use inet_ntoa for a
> human-readable string.
>
>   
>> I've also added the following line, immediately before the break; statement.
>>
>> printf("Backend %d\n",res);
>>     
>
> res is a pointer to a structure, but the results are good enough.
>
>   
>> I've then set pound to not run as a daemon.
>>
>> Ok the the following is the output I get.
>>
>> When connecting from host 10.190.66.66
>>
>> Host: 1111670282
>> Backend -1957071616
>>
>> When connecting from host 10.190.70.70
>>
>> Host: 1179041290
>> Backend -1957071616
>>
>> As a matter of interest I also added the latter printf statement just 
>> prior to the break; in the SESS_NONE section.
>>
>> When running with 'no sessions' I get the following output:
>>
>> Backend -2029771520   <- client connects to server 1
>> Backend -2029771008   <-client connects to server 2
>>
>> The sequence is obviously random.
>>
>> I hope thats what your after, if not treat me as an idiot and tell me 
>> what you want.
>>     
>
> That is almost what we need. If you could change the printing of
> from_host as described above it would be very helpful, if you could also
> show the value of pri it would be even better. The code would look a bit
> like this:
>
>     case SESS_IP:
>         /* "sticky" mappings */
> fprintf(stderr, "from_host: %s\n", inet_ntoa(&from_host));
>         addr = from_host.s_addr;
>         pri = 0;
>         while(addr) {
>             pri = (pri << 3) ^ (addr & 0xff);
>             addr = (addr >> 8);
>         }
> fprintf(stderr, "pri: %d\n", pri & 0xffff);
>         res = rand_backend(svc->backends, (pri & 0xffff) %svc->tot_pri);
> fprintf(stderr, "backend %ld\n",res);
>         break;
>
> I sincerely hope you don't think this is idiot-like treatment - we are
> certainly interested in your feedback and we would like to fix the bug
> rather than insult people.
>   

Hi Robert,

Ok I've made those changes, I now get the following output:

# ./pound -f /etc/pound21-dev.cfg

 From Host: 10.190.70.70
pri: 14838
Backend : 2081694976

 From Host: 10.190.66.66
pri: 14802
Backend : 2081694976

 From Host: 10.190.70.70
pri: 14838
Backend : 2081694976

 From Host: 10.190.66.66
pri: 14802
Backend : 2081694976

Hope that helps.

Many thanks

Simon


Robert Segall wrote:
> On Fri, 2006-08-18 at 11:53 +0100, Simon Slaytor wrote:
>   
>> Hi Robert,
>>
>> Ok, I'm no programmer so forgive me if this is wrong.
>>
>> I've added the following line to the SESS_IP section in svc.c, 
>> immediately following the pri = 0; line.
>>
>> printf("Host: %d\n",from_host);
>>     
>
> from_host is structure. Try printing out addr instead, as it contains
> just the information you want. Ideally use inet_ntoa for a
> human-readable string.
>
>   
>> I've also added the following line, immediately before the break; statement.
>>
>> printf("Backend %d\n",res);
>>     
>
> res is a pointer to a structure, but the results are good enough.
>
>   
>> I've then set pound to not run as a daemon.
>>
>> Ok the the following is the output I get.
>>
>> When connecting from host 10.190.66.66
>>
>> Host: 1111670282
>> Backend -1957071616
>>
>> When connecting from host 10.190.70.70
>>
>> Host: 1179041290
>> Backend -1957071616
>>
>> As a matter of interest I also added the latter printf statement just 
>> prior to the break; in the SESS_NONE section.
>>
>> When running with 'no sessions' I get the following output:
>>
>> Backend -2029771520   <- client connects to server 1
>> Backend -2029771008   <-client connects to server 2
>>
>> The sequence is obviously random.
>>
>> I hope thats what your after, if not treat me as an idiot and tell me 
>> what you want.
>>     
>
> That is almost what we need. If you could change the printing of
> from_host as described above it would be very helpful, if you could also
> show the value of pri it would be even better. The code would look a bit
> like this:
>
>     case SESS_IP:
>         /* "sticky" mappings */
> fprintf(stderr, "from_host: %s\n", inet_ntoa(&from_host));
>         addr = from_host.s_addr;
>         pri = 0;
>         while(addr) {
>             pri = (pri << 3) ^ (addr & 0xff);
>             addr = (addr >> 8);
>         }
> fprintf(stderr, "pri: %d\n", pri & 0xffff);
>         res = rand_backend(svc->backends, (pri & 0xffff) %svc->tot_pri);
> fprintf(stderr, "backend %ld\n",res);
>         break;
>
> I sincerely hope you don't think this is idiot-like treatment - we are
> certainly interested in your feedback and we would like to fix the bug
> rather than insult people.
>   

On Tue, 2006-08-22 at 09:17 +0100, Simon Slaytor wrote:
> Hi Robert,
> 
> Ok I've made those changes, I now get the following output:
> 
> # ./pound -f /etc/pound21-dev.cfg

I assume this is the config file with Session IP.

>  From Host: 10.190.70.70
> pri: 14838
> Backend : 2081694976
> 
>  From Host: 10.190.66.66
> pri: 14802
> Backend : 2081694976
> 
>  From Host: 10.190.70.70
> pri: 14838
> Backend : 2081694976
> 
>  From Host: 10.190.66.66
> pri: 14802
> Backend : 2081694976
> 
> Hope that helps.

It certainly does. What we have here is two clients who just happen to
hash to the same back-end, which, given you have only two back-ends, is
not really that surprising. The values you show are perfectly normal and
Session IP would work fine.

To test it: try connecting from a client with an odd network address
(10.190.70.71 would be a good candidate) and/or play with the priorities
of the two back-ends (4 and 3 to make a prime number for tot_pri).

> Many thanks
> 
> Simon

You're welcome. Thanks for your effort.
-- 
Robert Segall
Apsis GmbH
Postfach, Uetikon am See, CH-8707
Tel: +41-44-920 4904