Re: [Gnu-arch-users] back-building pfs updates

[Aaron Bentley]

On Sat, Feb 21, 2004 at 03:21:46PM -0800, Tom Lord wrote:
>     > Finding cachereves and continuation files currently requires at least 1
>     > server round trip for every revision that may (but almost always
>     > doesn't) contain a continuation file or cached revision.
> 
>     > But with http pipelining, I believe it could be reduced to two server
>     > round-trips:
>     > 1. List all revisions
>     > 2. Get all revision directory listings
> 
> Well, hmmm.
> 
> That sounds desirable.  What I _don't_ want is (non-trivial) logic
> above the archive.h layer that complicates the currently serial
> archive abstraction.

Reasonable.  There's a line to walk between having things too automatic
and too manual.  Too automatic, and tla starts making bad guesses about
what we really want, and doing the wrong thing.  Too manual, and we're
dependant on transport format eccentricities.

It seems the requirements are:
- Optimal behavior whether or not the underlying transport supports
  parallel operations/pipelining
- No serious impact on anything above archive-pfs


> So, I wonder if that kind of pipelining can't be done at archive-pfs
> or below --- with just a minimum of hints from the higher layers.

Right now, I keep coming back to the idea of a revision data struct:
struct rev_data
{
   t_uchar * prev_continuation
   t_uchar * prev_cache_rev
}

rev_data * get_revision_data (t_uchar * revision, t_uchar
* prev_revision, * next_revision)

I can imagine using library revisions as the next and prev revision,
since if there's no continuation revision in between, those would often
be the best choice as a data source.

>     > In my book, archive mirroring is too coarse a level of granularity.  I
>     > don't want to download jblack's integration, though I do want his
>     > contrib.  I don't want Robert Collin's barch stuff, just his
>     > integration.  I don't want scheme, just tla--devo--1.2.
> 
> The "limit" facility of mirroring already gives you that.

>     > But I don't want to make mirrors that aren't.  If I could mirror a
>     > version without pretending to mirror an archive, I'd be willing to
>     > consider it.
> 
> You can -- that's what the "[limit]" parameter in:
> 
>     update an archive mirror
>     usage: tla push-mirror [options] [from [to] [limit]]
> 
> is for.

My understanding is that I can only use one version of an archive.
Either it's the remote version, or a local one.  If my local mirror is
incomplete, I can't access those things my local mirror is missing.

So it seems to me that partial mirrors violate the notion that any
copy of an archive is interchangeable with any other.

>     > Continuations and cacherevs can happen anywhere, though.  And the actual
>     > data isn't huge; it's the latency that kills us.  That's why I want a
>     > two-round-trip approach, instead of an 0(n) approach where n is the
>     > number of revisions.
> 
> That's fine.   I just don't want to hair up the code above archive.h
> to achieve it.

Sure, I can understand that.

Aaron