Re: [Arx-users] RFC: arch protocol, smart server, and tla implementation prototypes

[Walter Landry]

Colin Walters <address@hidden> wrote:
> On Sat, 2004-01-31 at 21:33, Walter Landry wrote:
> 
> > The current situation can be vastly improved with just a smarter
> > client.  The current client is much more inefficient than it has to
> > be.  It should be able to download things in parallel, instead of
> > waiting for the result of one result before continuing on.  For
> > example, for a simple get, you do one or two directory listings to
> > find the latest revision, and then a whole lot of listing in parallel
> > to find the latest cached revision and continuation revisions.
> 
> First, this depends very much on the underlying transport.  I don't
> think FTP or SFTP are pipelinable.  HTTP/1.1 is, but it's read-only.
> Opening multiple simultaneous connections is pretty evil (server
> overhead, defeating TCP congestion control).
> 
> And moreover, note that there might be no cached revisions at all.  Some
> people expect that users of their archives create a local mirror with
> cacherevs themselves, or use a revlib.  It then becomes pure overhead to
> search through all the revisions looking for a cacherev, even with
> pipelining.  
> 
> A smart server could just say up front that there are no cacherevs, and
> the client could immediately start retrieving from the last continuation
> or import advertised by the server.

I agree that it is less efficient.  I'm just not convinced that it is
big enough to matter.  We're not talking about a lot of data.  We are
just stat'ing the files.

> > > o Execute post-commit hooks on the server when a revision is committed
> > > o Ability to have very flexible authentication and access control
> > > o Removing the need for a shared Unix account for multiple committers
> > 
> > These can already be done using arch-pqm or simple extensions to it.
> > You don't really have to define a wire protocol for this.
> 
> Yes, I know, having written arch-pqm ;)  But it's really a different
> beast.  arch-pqm is very indirect, and for some projects where you want
> to run the regression tests before *any* commit, that's an overall win.

If you want regression tests before local commits, why can't you use
local hooks?  I don't know if tla has them, but ArX has had them for a
while, and they weren't hard to add.

> But for a project where you just want to share an archive, it's
> basically overhead.  I like the flexibility and immediacy of direct
> access - for example, replaying just a few patches into the central
> archive instead of star-merging.  Could we add a "replay" command to
> arch-pqm?  Yes...but it's just not the same thing as doing 'tla replay
> ...; tla commit'.

Arch-pqm is currently not immediate (unless I missed something), but
that doesn't have to be the case.  As for the rest, isn't it just
syntactic sugar?

You could also argue that the commands should be separated, since tla
generally doesn't operate on shared repositories, while arch-pqm does.

> > Couldn't you do a poor man's version of this with a cron job?  It
> > would look over the logs, figure out which revisions are popular, and
> > cache those.
> 
> Sure.  It would just be easier with archd.

Definitely.  Just to be clear, I completely agree that there are some
benefits from having a smart server.  I am just not convinced that it
is the best way to proceed now.  It seems easier to improve the client
rather than defining a new protocol, making sure that the server is
secure, dealing with firewalls, etc.

Cheers,
Walter