Re: [Monotone-devel] mtnplain (aka dumb) revision order, epoch

[Nathaniel Smith]

On Fri, Jun 23, 2006 at 09:40:31AM +0200, Zbigniew Zagórski wrote:
> Hi,
> 
> Recently I recognized that plain (aka dumb) that is rather far from working 
> with
> mtn 0.26.
> On 0.27 it's better because it works on windows (automate stdio newlines 
> fixed).
> There is a problem with revision reordering which I almost fixed in my 
> sandbox.
> 
> This fix is to recognize all the certs received by Feeder and
> queue them accordingly to the type:
> - fdata   - write them immediately
> - fdelta  - write them immediately
> - pubkey  - queue
> - rdata  - queue
> - rcert  - queue
> Then when everything is fed to the Feeder:
> - write all 'pubkey' - this probably can go unordered
> - write all 'rdata' (*)
> - write all 'rcert'
> 
> (*) The hack is with writing rdata in correct order. Currently after writing
> each rdata packet mtn 'stderr' is checked if contains:
>    missing prerequisite revision 'REVID'
> and if text is found then this revision is queued as a child of REVID. Then 
> when
> REVID is written then all its children are queued again.
> 
> This is almost correct but certainly that 'stderr' matching is awfully evil. 
> Yes
> i know.
> Question: How to obtain parent REV(s) from rdata packet without using 'mtn 
> read'
> ?

I thought that someone else already fixed this in the current head of
the .dumb branch -- the better solution is to make sure that when we
write things out to the DATA file, we write them in the correct order,
so that later on they can just be read in directly like they are now.

It's possible that this is in fact fixed, but you're trying to pull
from a "dumb" repository that was created before this was fixed, so
the packets are still in the wrong order.

I suggest doing a push into a fresh dumb repository, and then pulling
from that, and seeing if that fixes the problem.

> Second problem.
> After finishing packet reordering I found that 'mtnplain' 'pull' to a clean
> database fails.
> This error was after following steps:
>  1. 'mtnplain' -d DB.mtn push SOMEPLACE
>  2. mtn -d DB2.mtn db init
>  3. 'mtnplain' -d DB2.mtn pull SOMEPLACE
>  4. mtn -D DB2.mtn pull file:DB.mtn
> 4th step was to check if all data were transfered by mtnplain if yes - netsync
> should transfer nothing; but apparently mtn failed with following error
> ------------
> |06:07:16#23|address@hidden|..umb/plain-zbigg|$ mtn -d DB2.mtn pull 
> file:DB.mtn
> 'net.*'
> mtn: connecting to file:DB.mtn
> mtn: finding items to synchronize:
> mtn: certificates | keys | revisions
> mtn:          162 |    3 |        54
> mtn: warning: error: Mismatched epoch on branch net.venge.monotone.dumb. 
> Server
> has
> 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client has
> '0000000000000000000000000000000000000000'.
> mtn: peer file:DB.mtn disconnected after we informed them of error
> mtn: warning: protocol error while processing peer stdio: 'received network
> error: Mismatched epoch on branch net.venge.
> monotone.dumb. Server has 'e748431d4147df17a39df80f0c2d72c9d2fa9fb2', client 
> has
> '0000000000000000000000000000000000000000'.'
> mtn: bytes in | bytes out | certs in | revs in
> mtn:      129 |       235 |        0 |       0
> ------------
> 
> Hmm, what is epoch and what should I do while exporting/importing to preserve
> epoch of a branch?

Ah, good question!  The documentation on epochs is here:
  http://venge.net/monotone/docs/Rebuilding-ancestry.html
(It could probably be more clear.)

What needs to be done to support them in mtn-dumb is:
  1) create a new packet type for epochs
  2) for each branch in the db, add this packet to the merkle trie
     being synced.  Make sure to put these packets as early as
     possible; just like revisions should come before certs, epochs
     should come before revisions, and even before files.

(1) is best done in monotone itself.  I'll put it on my list to do,
but I already have a bunch of "do this immediately!" things on my
list, so if you want to do it, or anyone else out there does, that
will likely be quicker :-).

You could also, for now, implement (1) yourself -- basically make up a
special format that mtn-dumb will recognize.  Then generate packets by
isng "mtn ls epochs", and when reading in packets, check for the
special format, and if you see it then first use "mtn ls epochs" to
make sure that the local db has no epoch for the branch in question,
if it does have an epoch then error out, and if it doesn't then call
"mtn db set_epoch" of passing it to "mtn read".  (You might need to
queue the set_epoch calls up until the end, so as not to to fight with
"mtn read" for locks on the database.)  (This is exactly how an
in-monotone implementation of epoch packets would do, except with
fewer caveats.)

> PS. I think that 'plain' is better name for 'dumb'. And my proposition is
> to name the tool is 'mtnplain'.

Hmm, I'm not sure.  There are two problems:
  1) "dumb" is sort of a stupid name, but it has a lot of history
     behind it :-).  The standard word for using HTTP/FTP/etc. as a
     VCS transport has been "dumb" for many years now, and it's been
     used by a lot of different people (starting with the Arch
     people).
  2) "plain" is very ambiguous.  For instance, surely "plain" monotone
     is what you get when you go to
        http://venge.net/monotone/downloads/
     right?

> PS2. Here is latest version of 'mtnplain'
>   http://zzagorski.strony.wi.ps.pl/mtnplain/mtnplain-0.0.3.zip

That's awesome!  Really appreciate your work on this.

Want to send me a key, so you can use the venge.net server to
distribute your work?

-- Nathaniel

-- 
So let us espouse a less contested notion of truth and falsehood, even
if it is philosophically debatable (if we listen to philosophers, we
must debate everything, and there would be no end to the discussion).
  -- Serendipities, Umberto Eco