Re: [DotGNU]Our blindspot

[Chris Smith]

On Saturday 09 February 2002 08:45, Gopal.V wrote:
>       So will a distributed LDAP (model) server do ?. With multiple
> top levels ,ie your host has settings for each TLD ie "gnu.*" redirects
> to an IP, "perl.*"...........we may even imitate the newsgroup
> style .....(comp.* style) and make data into standard DTD's.


This is probably the way to go.
We must make sure that there is a way of mapping multiple IP's to a resource 
to handle multiple machines serving the same resource.
Also these IP's must be able to come and go.

I've recently done some work where this was an issue and I've implemented 
quite a nice system to solve the problems.  It's not perfect, and runs in a 
very closed world, but works a treat.

If you want to know more about the model implemented, read on, (it may at 
least be a source of ideas)....

My customer has a server containing a set of resources, and they want some 
sort of HA and load sharing (it's beginning to creak).... (the resources on 
the server do not change very often).

So I built a collection of these servers so that they were identical.  When 
each server starts up it causes it's IP address to be added to the internal 
DNS.  It does this via SNMP.  When a server shuts down, it removes itself 
from the DNS via SNMP.  (When a server detects that it is failing, like when 
it starts to produce critical applicaiton errors, it removes itself from the 
DNS as well).

The client interface to these servers was modified to include transparent IP 
caching and round robbin.  The client contacts the DNS and gets all IP 
addresses for the service/service it is interested in.  It does this no more 
than once every 20 seconds (configurable).
It scrambles the order of the IP's in it's list and for each request it needs 
to make, chooses each one in turn.
If it finds it cannot contact one of the IP's for whatever reason, it marks 
that IP as unavailable in it's internal cache and moves onto the next.  If 
there is only one IP in the cache that is available, it does not mark it 
unavailable in a failure condition (otherwise the client would have no where 
to go - this was inportant for my customers app - not strictly necessary...).

When the client next refreshes it's IP cache, any 'unavailable' IP's persist. 
The only way to get an IP that is unavailble back into an available state is 
to take it out of the DNS (thus the client will forget about it) and then put 
it back in.  This is fine, as failure to contact a server is fatal and 
requires human intervention anyway.

This basic mechanism allows the customer to add and remove servers at will to 
the pool, and have this activity completely hidden from the client apps, who 
themselves transparently manage failures.


Now the dotGNU situation is that we don't want to have a single 'DNS'.  
However, the above model might still work if the client, on detecting that a 
server is not responding, updates the 'Service Discovery Node' (Oooh SDN :o)
that it got the IP address from and tells it that it is unavailable.  
Basically some kind of feedback.  That'll stop it serving the IP address of a 
dead or disconnected server.  Where services are distributed, there will be 
other IP's available.  SDN's need to propogate up/down info between 
themselves, but if clients are intelligent as to the IP's they contact then 
this does not have to be super quick.

This feedback is of course open to attack, but when a SDN dishes out sets of 
IPs, it could also hand out a feedback ticket ID that is valid for a short 
time, during which it will accept an update as to the status of the server at 
that IP.  This may also allow load metrics to be passed back, enabling proper 
distributed load balancing - or is that just dreamware??????

Regards,

Chris

-- 
Chris Smith
  Technical Architect - netFluid Technology Limited.
  "Internet Technologies, Distributed Systems and Tuxedo Consultancy"
  E: address@hidden  W: http://www.nfluid.co.uk