Re: [Monotone-devel] Push can pretend it is successful while it is not

[Nathaniel Smith]

On Wed, Aug 17, 2005 at 10:44:35AM +0200, Marcel van der Boom wrote:
> 
> On 16 aug 2005, at 15:01, Nathaniel Smith wrote:
> 
> >Hmm.  It looks like what happened is that it _did_ run successfully
> >(in the sense that it successfully transmitted all information), but
> >then before the server finished committing the transaction to disk it
> >ran into a little problem:
> 
> Yes, should the "transaction" not be rolled back then?

If this happened in the client, the transaction would have been rolled
back.  The server, though, is a single-threaded process that
multiplexes multiple concurrenct connections.  It only has a single
connection to the db, and that connection can only have a single
transaction open at a time.  So we can't put whole sessions inside
single transactions, because there may never be a time when there is
not some session active, which means we would never ever commit.
Which would be bad.

So instead, the server commits after it processes each packet.  Since
it's also careful to write in the correct order (we hope!), the worst
this can cause is some unreachable junk, which is harmless and
apparently unavoidable.  (And will speed up the next push that
succeeds in passing on this stuff.)

> >The other bug is that apparently netsync can finish successfully
> >before all data is written to disk.  We knew this, actually... hmm.
> >The netsync "goodbye" phase is just a way of determining that
> >everything that was supposed to go over the wire has, in fact, gone
> >over the wire, so there's no way in there ATM to even confirm disk
> >writes.  Could you file a bug about this on Savannah?
> 
> http://savannah.nongnu.org/bugs/index.php?func=detailitem&item_id=14157

Thanks.

> >>3. monotone db check came up with all sorts of reference errors  
> >>missing
> >>
> >
> >Could you give more details?  Because...
> In the process i did not save the initial output of db check, so here  
> it goes from memory, sorry bout that.
> 
> There were 200+ errors saying that a file was missing (the revision  
> going in happened to change/remove a lot of files) and a couple of  
> errors mentioning unreferenced stuff.
> The unreferenced stuff is annoying but doesnt worry me, the missing  
> files errors let me to think that 'metadata' was stored but actual  
> changes were not.

Hmm, it's also possible that the manifest got saved, though the
revision didn't.  Though that shouldn't be possible, either.  Though I
guess we don't actually have any tests for the (elaborate and
in-need-of-rewrite, to become non-recursive...) packet ordering
code...

Actually, this is what most likely happened, since 'db check' doesn't
look for files referenced by revisions (it makes sure that the
manifest referenced exists and is correct, then checks the manifest).

> >...if this is true there's another bug; but it doesn't sound like it
> >was quite true, from your mention that viewmtn didn't show the
> >revision.
> it wouldnt show it in either case, would it?

I don't know; I guess it depends on how viewmtn handles assertion
errors, whether it would show a revision whose manifest was missing...

> >What _should_ have happened is that the files and perhaps
> >manifests were stored to the db, but the revision (the "metadata") was
> >not; monotone stores things in this order on purpose, so that you
> >might have some unreferenced (and thus pointless) data sitting around
> >your db, but you'll never have any references to data that just isn't
> >there...
> Right, the error messages made me believe the reverse had happened  
> (which *would* worry me)
>  There is probably no reliable way for me to reproduce this at this  
> point though.

Unfortunately, yeah.

-- Nathaniel

-- 
"...these, like all words, have single, decontextualized meanings: everyone
knows what each of these words means, everyone knows what constitutes an
instance of each of their referents.  Language is fixed.  Meaning is
certain.  Santa Claus comes down the chimney at midnight on December 24."
  -- The Language War, Robin Lakoff