[Gnu-arch-users] Re: arch commit on large trees ?

[Philippe Moutarlier]

On Thu, 2005-08-25 at 22:47 +0200, Matthieu Moy wrote:
> "Philippe Moutarlier" <address@hidden> writes:
> 
> > baz inventory -s | wc -l
> >
> > 22133 files
> 
> With that size of project, you should split your project into several
> smaller ones, and use, for example, configurations (see baz
> build-config) to manage the nested projects.
> 
> I'm not 100% happy with the way configuration work, but it's probably
> the best way to scale up with arch.
> 

I am trying to be as little disruptive from the way things are organized
in CVS right now. I'll give a look to see how difficult it would be. 

> > There are 2 delays : the first one is before even looking for the
> > revision. Why ?
> 
> I don't know the exact algorithm, but this may be due to baz doing an
> inventory of your local tree. Roughly, it does a "find + regexp
> matching" in all the files of your tree. It happens to be quite slow
> on large trees.
> 

I am not sure why we want all that to happen before we even look for for
the revision in the library and then before we start comparing the tree
with it. Is it really necessary ? 

> > Can we turn off the integrity check ? In general we have to trust our
> > own file system ... (and back it up !) otherwise little could be done.
> > In case of a purely local branch, I don't see the benefit of this
> > check.
> 
> You will see it if you happen to get a corrupt revision library once
> in your life. Arch just checks that you didn't modify the files in the
> revision library. It's checking the chair-keyboard interface as much
> as the filesystem or anything else actually.
> 
> Since the changeset uploaded in the archive is based on the content of
> the revision library, it means that if you have a silently corrupted
> revlib at revision N, all the content of your archive after revision N
> is potentially unusable.
> 
> > I am a little disappointed that it seems  arch doesn't quite scale up to
> > that project size.
> 
> By curiosity, which other RCS did you try, and what performance did
> you get?
> 

the good old CVS.  And yes, checkout is slow with CVS but not commit,
because it allows a per-directory commit which is not supported (or is
it ?) by arch. 

Thanks for your patience :-)

Philippe

> > That was what I heard around but I wanted to give a try for myself.
> > I am afraid there is no way I could get my guys to buy it, which is
> > too bad.
> 
> GNU Arch does have problems with really large trees (on the other
> hand, it's extremely bandwidth-saving and puts very little load on the
> server, so you can host a big number or medium projects on a
> reasonable server). Bazaar 1.x improves some of the points, and you
> may want to have a look at the future Bazaar 2.0, AKA Bazaar-NG, who
> will have dramatically improved performances (but you'll have to wait
> a few months before being able to use it for production).
>