Re: [Monotone-devel] Re: Fwd: Re: FreeBSD's requirements for its future VCS

[Nathaniel Smith]

On Sat, Nov 11, 2006 at 11:38:24PM +0100, Lapo Luchini wrote:
> Bruce Stephens wrote:
> > I'm not sure how you'd get rid of a file.  I guess you have to take
> > the revision that added it, commit another one that doesn't add it,
> > then rebase all its descendents?
> 
> What about adding a "forbidden_content (id, reason, certificate)" table
> to the DB and:
> a. delete the files with version "id" from the file table
> b. delete the delta which "produces" that version from the delta table
> c. adjust the delta of the following version to skip the deleted one
> d. spread this table like every other

It is step (d) that is difficult.  We don't "spread tables" at all;
monotone's synchronization works by first determining a list of facts
that are only know locally, and then sending them to the remote side.
Note that this assumes that facts only accumulate; we never send
"anti-facts", for a whole mess of reasons.  (How do you figure out
what anti-facts need to be sent?  Who do you trust to create
anti-facts?  Do you really want to add a whole new layer of
complexity for a rarely used feature?  Well, maybe.  But maybe we
should talk investigate our options some too.)

The two big questions about obliteration are:
  1) Do you want the data to simply be unavailable, or do you want to
     remove every trace of its existence?  Approaches like yours that
     leave the file hash in the manifest mean that even if the file
     itself is gone, it is always cryptographically provable that it
     _once_ was available.  Conceivably, you might be obliterating a
     file to preserve trade secret protection or to otherwise protect
     yourself legally, and leaving this kind of trail behind would be
     bad.
  2) How smooth does the obliteration process have to be?  Is it
     acceptable if mirrors require manual intervention?  Is it
     acceptable if workspaces have to manually recreated?

I would be curious to know what the FreeBSD folks's answers to these
questions were.

For comparison, IIUC svn's support of obliteration (by doing
svndump/svndumpfilter/recreating the repository) breaks workspaces,
because you have essentially created a whole new central repository.
And of course they don't have mirrors.

This basic technique generalizes pretty easily to monotone.  All you
need is some sort of command to "rewrite" history with changes.  It
should take a bunch of revisions, and create new revisions that have
the same ancestry graph and trees, except maybe with some changes.
Maybe leaving out all trace of certain files, or leaving out a few key
commits, or whatever.  Then when you need to obliterate something, you
first rewrite history as if the bad stuff had never happened.  Now
your problem is all the mirrors on everyone's hard drive that still
have the old bad history in them.  To prevent that bad stuff from
coming back, you can just bump your "epoch" on the relevant branches
-- this will require everyone who has bad stuff to do a fresh pull.
And, of course, workspaces all need to be blown away and recreated,
because they point to revisions that no longer exist.

This does great on point (1), and terrible on point (2).

Another approach, assuming we get trust stuff worked out, would be to
replace the use of epochs with explicit trust decisions.  The admins
say "actually, don't trust any of the certificates on the bad old
revisions; instead, trust the certificates on our fancy new
revisions".  With this approach, mirrors work smoothly -- they just
pick up the new trust rules, and while they still _have_ the old bad
stuff, they will no longer pass it out to anyone else.  Workspaces are
still broken, though.

-- Nathaniel

-- 
Damn the Solar System.  Bad light; planets too distant; pestered with
comets; feeble contrivance; could make a better one myself.
  -- Lord Jeffrey