Re: [Monotone-devel] Re: netsync question

[Markus Schiltknecht]

Hello Bruce,

On Sat, 2006-04-29 at 12:54 +0100, Bruce Stephens wrote:
> The deltas aren't (generally) reversible.  A delta from A to B is a
> sequence of copying byte sequences from A and inserting new byte
> sequences.  (Very like xdelta.)

That's what I call an 'Aha-effect'! Thank you.

> Currently that's not how things work.  Right now if you have a
> repository then you can verify everything in it, right back to its
> beginning.  
> 
> If you had only the last few revisions, you'd need to take something
> on faith.  If I got some late revisions of net.venge.monotone from a
> corrupted (or malicious) repository, then it might be that I couldn't
> fill in earlier revisions from venge.net, and I wouldn't be able to
> tell without checking.

The point is, most users don't care about a completely consistent
history. They want to check out the latest sources. That's what netsync
should be optimized to do, IMHO.

That other users, namely the committers often do care about a consistent
history is another thing which netsync should guarantee, too.

If two syncing nodes agree on a common ancestor (which they both have in
their db) I think it's guaranteed that exchanged revisions are sane
(given that transmission is failsave).

Again, this would probably require forced consistency checks before
commit, push, sync or serve. Full consistency checks in turn can only be
made with the complete ancestors history.

> I'm not sure whether that actually matters; maybe that would make a
> coherent system, too.
> 
> But it would be different.

Yes, indeed. But can it be made as save and coherent as we need it to
be?

Some further brainstorming:

A Node serves a repository, another node already has some data and does
a subsequent pull. They agree on revision A to be the common ancestor.
Revisions B, C and D are new on the serving node.

Uppon client request only the latest revision D gets transfered (with an
additional fulltext or an A->D forward delta). On the pulling node
revisions B and C are missing. That node should keep a hint about that
gap and how to fill it (what node was serving it).

The user on the pulling node now checks out revision D, does some work
and wants to commits a revision E. Should that be allowed? We know its
ancestor, D is considered good by the origin node.

If the user commits and pushes his revision E back to the origin node
that poses no problem. The origin can easily check if E is faulty or
not. So the user could continue to live (and work) with that gap.

But what if he wants to push his repository to another user? He could
eigther fill the gap or also send the gap hint. The other user would
then get a gap with a hint like 'node X says I could fill that gap
asking node Y'.

That seems complicated. To prevent that we'd have to prevent nodes from
serving repositories with gaps and only allow pushing to the node which
knows all revisions in the gap.

Well, dunno if that's worth it... thank you for your patience reading
this ;-)

Markus