[Monotone-devel] speed up the initial pull

[Markus Schiltknecht]

Hi,

I'm replying to several people, starting a new thread. Hope that's okay.

On Fri, 2006-04-28 at 13:20 -0400, Ethan Blanton wrote: 
> I think we're talking about two different things here, in several
> points along this thread.  :-)

I was hoping I made the distinction clear. Obviously I did not, sorry.
I'm envisioning netsync handle transmission failures, which can occur
during transmission. But I would like to check and take care of other
failures at other times, not necessarily during transmission.

On Fri, 2006-04-28 at 10:43 -0700, Justin Patrin wrote: 
> Here's a possible solution which I have just thought up, spur of the
> moment. Include a --no-consistency-checks option to stop the hashing
> and consistency checks on pull, but keep track of this in the DB.
> Allow the user to check out a working copy and work with it, then when
> any other event happens (I'm thinking commit, push, or allowing pull
> via mtn serve) monotone will do the obligatory consistency check. It
> needn't be on every revision in the DB, only the ones which are
> connected to the current one by ancestry.

That's very close to what I'm envisioning. Justin correctly adds checks
before push, sync and serve. Good catch.

On Fri, 2006-04-28 at 10:48 -0700, Nathaniel Smith wrote: 
> So, a modest proposal -- maybe some of the energy in this thread would
> be good to direct towards speeding up the netsync we already have?
> Remember, right now, _no-one knows_ whether we can make netsync fast
> _and_ safe -- maybe it can be done!  Wouldn't _everyone_ prefer that,
> if we can manage it?

Complete ACK.

> And, you know, if we try and fail, then of course we'll consider other
> options.  But right now it feels like we're rejecting the best
> solution, because we're worried it _might_ not work.

I've never argued to reject the best solution (tm). I even see my
proposal more as a replacement for the 'download-the-db-via-http'
workaround than as a competitor to netsync.

As soon as netsync manages to transfere repositories in a reasonable
amount of time I'm more than willing to drop any workaround. But until
we are there we can either dissappoint early-adopters or accept some
sort of workaround.

On Fri, 2006-04-28 at 19:19 +0100, Bruce Stephens wrote: 
> I'd guess reconstructing the bytes to check (applying the xdelta
> patches and things) is vastly more costly.

Up until now I thought 'consistency checking takes so much time' was the
answer to the 'why does pulling it take so long?' question.

Honestly, I don't know and if you are right, then of course such a
workaround is pretty useless.


My conclusion: I want to better understand how netsync works and I need
to do some profiling. I'll let you know...

Regards

Markus