[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gnu-arch-users] Avoiding ancestor scan during get with revision lib
From: |
Aaron Bentley |
Subject: |
Re: [Gnu-arch-users] Avoiding ancestor scan during get with revision library |
Date: |
Thu, 06 May 2004 15:45:20 -0400 |
User-agent: |
Mozilla Thunderbird 0.5 (X11/20040309) |
Tom Lord wrote:
> From: Aaron Bentley <address@hidden>
> Agreed. It's too damn slow. But for the moment, it seems the latency
> issues are inevitable. Using pipelining, parallel downloads, a smart
> server, or changing the archive format could improve matters.
It can be solved more simply than any of that.
By "it", I mean the archive scanning. Once we know where the cacherevs
are, sure we can use 'em. But I'd like to see the scanning sped up. On
DSL, you can still wait minutes before it even *starts* downloading.
In this case, given a very recent cacherev and no recent ancestor in
the revlib, tla should give up on hard-linking fairly quickly and just
build a non-shared revlib tree. The exact point at which to give up
is something we can pick a default for and let people tweak with a
library-config parameter.
Actually, what I'd like to do is replace almost all of library_add()
with build_revision()*. The reason I haven't done that yet is that I'd
like to get the backbuilder to use the changes --link functionality to
hardlink anything that isn't already hardlinked, so long as it has some
kind of relation in the revision library. Once that's in place, we
won't have to trade storage space for download speed.
It's also something we can choose
automagically if we start recording file sizes in archives (and this
is like the N+1th reason we have for doing so).
So actually, it looks like the Nth reason (speeding up revision
building) to me, but definitely desirable.
Something that would be quite valuable in this would be non-volatile
ancestry data in the tree. Patchlogs are almost right (and I've got
tools that use 'em), but since patchlogs can be removed (e.g. by replay
--reverse, or by your good self), we can't rely on them in every case.
Local ancestry data would make it easier to
1. make revlibs extremely efficient
2. build revisions from their siblings (build foo from bar, when both
foo and bar are derived from baz)
3. make star-merge cross tag boundaries when seeking a common ancestor
* okay, not actually arch_build_revision(), but a close relation that
does hardlinking when it can.
Aaron
--
Aaron Bentley
Director of Technology
Panometrics, Inc.
- [Gnu-arch-users] Avoiding ancestor scan during get with revision library, Richard Curnow, 2004/05/06
- Re: [Gnu-arch-users] Avoiding ancestor scan during get with revision library, Aaron Bentley, 2004/05/06
- Re: [Gnu-arch-users] Avoiding ancestor scan during get with revision library, Tom Lord, 2004/05/06
- Re: [Gnu-arch-users] Avoiding ancestor scan during get with revision library,
Aaron Bentley <=
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Stefan Monnier, 2004/05/06
- Re: [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Miles Bader, 2004/05/06
- Re: [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Aaron Bentley, 2004/05/09
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Miles Bader, 2004/05/09
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Aaron Bentley, 2004/05/09
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Miles Bader, 2004/05/09
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Stefan Monnier, 2004/05/10
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Aaron Bentley, 2004/05/10
- [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Stefan Monnier, 2004/05/10
- Re: [Gnu-arch-users] Re: Avoiding ancestor scan during get with revision library, Andrew Suffield, 2004/05/11