[Gnu-arch-users] Re: arch and linux 2.7

[Andrea Arcangeli]

Hi Pau,

On Tue, Oct 28, 2003 at 11:38:46AM +0100, Pau Aliagas wrote:
> 
> Does anybody think that we should try to do something RSN to promote the 
> adoption of arch in the development of the linux kernel? Linux 2.7 is 
> about to appear and it would be great if new developers could adopt arch 
> as their default SCM.
> 
> We have most of the things in place and, maybe, only a little effort and 
> strong support from the current developers already using it would increase 
> the acceptance significantly.
> 
> The steps to take, a grosso modo, would be:
> 
> -stabilize the tla commands
>  * finish the rename of commands (what-changed -> changes, etc).
>  * update the tutorial/manual accordingly.
> 
> -add performance hooks addressed to expert users (sorry for the 
>  self-promotion, it's not on purpose :)
>  * sparse revision libraries
>  * hard-linked trees
> 
> -master archive (or repository):
>  * build a master archive with the maximum detail, maybe imported from 
>    bkcvs using the available tools, maybe imported from available 
>    detailed patches. We already have one with a release/pre-release level 
>    of detail, but it would be nice to have the maximum detail.
>  * group the tools to keep in sync with the original source (bkcvs, 
>    patches...)
> 
> -write a HOWTO-arch-linux-kernel:
>  * explain available optimizations to speed up things
>  * explain how to work with the tools to keep in sync with the original 
>    source (if you want to have your direct master archive).
>  * add examples of people already using arch with the linux kernel.
> 
> -try to approach several developers (maybe the current kernel developers 
>  using arch would be the best fitted for this) to expand its use.
> 
> Almost everything is already in place, maybe writing a good how-to is the 
> only part missing, but it's only a matter of mixing a few emails and 
> voilà!
> 
> Too early? Don't care? Comments?

starting with the "master archive" sounds a reasonable approch. I
believe it's already feasible to use arch for it. Once that is online,
it will be more natural to move to arch. I doubt suggesting using arch
on the l-k list for mainline kernels is a good idea, you know last time
I made a proposal not to use bitkeeper for 2.4 to Marcelo, the thread
ended with Linus asking me to go away from the list. we should only base
ourself on the open cvs, I will advertize the arch repo in my sig too
then, I will sure suggest it privately to my SUSE fellows (infact I
already did), but potentially bothering Linus and all the other
bitkeeper supporters sounds not a good idea, especialy now that arch is
still inferior to bitkeeper in various areas.

I don't worry about the cvs going away, if it will happen we will have
much more serious troubles than arch going out of sync.

On the technical side I believe many things should be changed to be
really optimal in those high end usages. I currently feel without
something like a superpatchset functionality, checking out and
downloading so many tiny tar.gz will be very inefficient compared to cvs
but we should try: build the repository and compare it with cvs
performance in local checkout, remote checkout and size of the archive.
I'm fairly sure arch will lose big at the moment.

We'll also need a "search" functionality like cvsps, but that one can be
implemented later.

The next most important thing to address is probably to make it as close
as cvs in terms of download speed (I mean with the full granularity
cached locally not by fetching the cacherev) and in terms of archive
size (uncompressed vs uncompressed [cvs is compressed too during network
transfer]). After all the information stored inside is the same, if
something arch should be much more compact if really optimized since it
can understand the renames.

The duplication in the patchset that leads to much bigger archive
annoys me a lot, I would prefer a binary format that doesn't even waste
bytes, using ascii in the repository is a waste. Infact I believe with
today powerful cpus we could try to use the spaces as separators not the
newline, to reduce even more the size of each diff. I would like an
optimized database, not those flood of tiny tar.gz with replication
inside and with very bad compression ratio because too small.

I can't care less about the cpu cost, I'd be fine with tla implemented
in python or java, I don't care, what I care is the format of the
database being compact, and efficient to checkout and to fetch through
the network and I can still see quite an huge room for improvements in
this area, but as said above those can be deferred for long term.

thanks!

Andrea - If you prefer relying on open source software, check these links:
            rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/
            http://www.cobite.com/cvsps/