Re: [Gnu-arch-users] archive storage format comments on the size

[Andrea Arcangeli]

Hi Mark,

On Tue, Sep 30, 2003 at 09:35:40PM -0500, Mark A. Flacy wrote:
> >>>>> "Tom" == Tom Lord <address@hidden> writes:
> Tom> 
> >> From: Andrea Arcangeli <address@hidden>
> >> I don't see why you answered to the superpatchset idea with the
> >> "don't send noise" email. It just doesn't make sense.
> Tom> 
> Tom> Please consider that for the audience that most matters, your
> Tom> superpatch idea has been lost in the volume and unrelated content of
> Tom> your posts.
> 
> Not that I'm a member of *that* audience by any stretch of the imagination,
> but I've stopped reading Andrea's posts a long time ago.  As of late,
 
Not everybody probably can ever be interested in where I'm looking to
drive arch into. So I don't expect every user in the list to be interested
in my emails (even if I would been more ordered and accurate in sending
comments), that's reasonable, the tool already works fine for the
majority of you reading this email and I see that.  But if it will work
for me me too, then it will sure still work for you, and I'm fine that
things like tagline exists as far as you give me the option not to do
the work I want arch to do for me trasparently.

So I don't see problems, and I'm optimistic because the design looks good
and all the showstoppers seems solvable problems so far, without
altering the basic design of arch (even the superpatchset idea wouldn't
alter the basic design, it's only a change in the format of the archive
that will allow to fetch extract and store the info more efficiently).

Pau already implemented the get --hard-link option from revlibs, that
was really fast!

Now the next problem is that to generate revlibs for 15848 files in each
revision lib, and 13000 revisions, I would need to create 200 million
files. Actually they're not all new inodes, but my biggest fs has only 3
millions inodes free and I probably would run out of blocks first ;).

So the next problem to fix is that revlibs have to be changed to be
generated only for a single revision or I probably can't appreciate the
power of the get --hard-links ;). Then during checkout arch should
search the nearest revlib and hardlink from it and go backwards or
forward with patches, it should threat revlibs like cacherevs.  Pau
already nicely offered to have a look at this too ;). Keep up the great
work Pau!

With the l-k bkcvs import, we're probably the first ones trying to drive
arch to these real big scenarios, I seriously doubt anybody generated
anything close to 200M files (though not literally inodes) of revlibs
yet.

Thanks everybody for all the help so far,

Andrea - If you prefer relying on open source software, check these links:
            rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/
            http://www.cobite.com/cvsps/