Re: [Gnu-arch-users] archive storage format comments on the size

[Andrea Arcangeli]

On Mon, Sep 29, 2003 at 03:09:44PM -0700, Tom Lord wrote:
> Is it desirable to add a mechanism that records "gone but not
> forgotten" logs?   Perhaps so.... but it's tricky (supposing, for
> example, that I later do something that would revert and remove some
> of those logs if they were still around).

I think the ideal is to have a way to tell "I don't think I need to
access between base-0 and patch-2000  for a very long time", the tla can
transparently ungzip all the tar.gz  patchsets, and create a superpatchset, that
is the .tar.gz of the directory base-0 - patch-2000, of the uncompressed
patchesets. That will be able to crunch the 30M into 2M of space,
without losing anything, and it'll still work trasparently the few times
you need to access below patch-2000 to look back.

When you look back, you can unpack the big patchset and look back
trasparently if needed. It's worthwhile to add an option that says "run
slow but don't take too much space (or memory if in /dev/shm)", so you
can choose if unpack the whole archive at once (in a temporary directory
or in /dev/shm for full speed ram operations), or if to extract the
patchsets one by one from the achive, so you can still work despite
you've less free space than the whole size of the superpatch
uncompressed.

When you need to access below patch-2000 likely it's because you want to
do the equivalent of a full checkout or a cvsps -f fs/exec.c, so you
need to access all the patchsets anyways, so the full uncompress should
be optimal anyways (probably faster than the single unzip of the tons of
different .tar.gz). In short I believe the superpatchset it's very
preferable not just for space compression, but for checkout performance
too. Oh, it'd better be in bzip2 format (possibly selectable optionally,
that will crunch another 20% of space or so).

still I'd really like to shrink the _uncompressed_ format of the
patchset as much as possible first. Then we'll get the final benefit by
using a superpatchset tarball as outlined above.

Andrea - If you prefer relying on open source software, check these links:
            rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/
            http://www.cobite.com/cvsps/