Re: package.el + DVCS for security and convenience

[Stephen J. Turnbull]

Ted Zlatanov writes:
 > On Mon, 07 Jan 2013 11:03:07 +0900 "Stephen J. Turnbull" <address@hidden> 
 > wrote: 
 > 
 > SJT> Ted Zlatanov writes:
 > >> I'm actually suggesting that the GNU ELPA maintainers (note the "GNU
 > >> ELPA" part here, this is not any ELPA maintainer) should review and sign
 > >> *every* commit to the GNU ELPA.
 > 
 > SJT> I have no idea what you think you're proposing.
 > 
 > I hope that doesn't make my proposal less ideal.

If I'm not alone, it certainly does.

 > SJT> Security reviews are expensive; I doubt you'll have anybody willing
 > SJT> to maintain GNU ELPA after a couple of months of that, unless you
 > SJT> pay handsomely, or you enlist a maintainer per package or so to
 > SJT> reduce the burden on individual maintainers to a manageable level.
 > 
 > Trust is expensive.

You're scaring me again.  Trust is cheap, but risky.  That's not the
same.

 > The alternative is to say "trust this code, though we don't know
 > what it is."

Your use of the unique quantifier is once again inappropriate.  "We"
does not have to be an undifferentiated blob; it can be personalized.
You could, for example, have each volunteer provide a separate
signature as they review the code.  More reviews, more trust -- maybe,
unless more reviews means each one is less thorough.  Identification
of the reviewer means trust can be a function of the reviewer.
Authors don't make the best reviewers, but they do know the code, and
an author signature is a certificate of authenticity.

In general, having signatures that identify (a) the stage at which the
code was signed and (b) the authority doing the signing are useful
inputs to (some) users' trust functions.  I don't know if it's worth
it, but it certainly proves that your strawman is not the only
alternative.

 > That's the current state of affairs.

And, to a large extent, it will be the state of affairs under your
proposal, unless your "FSF and GNU volunteers" do a certain amount of
work on *every* commit to GNU ELPA.  The only thing that signing (by
itself) proves is provenance.  If it's signed, you know the signer
touched it.  That is valuable information, even though it doesn't
prove the code is clean.  If you actually do reasonably thorough
reviews, though, you will end up with a large backlog of pending
commits unless you have a lot of volunteers.  (At least one hopes that
there will be a lot of code committed to GNU ELPA, right?)

 > There is certainly review of code that goes into GNU Emacs itself.  A
 > security exploit would be caught quickly (I hope, based on previous
 > cases like the file-local variable exploits).

Uh, yeah ... maybe.  AFAICS much code that goes into Emacs doesn't get
reviewed by someone other than the author/committer, most code review
that is done for Emacs itself is hardly security-oriented, and the
exploits were caught after the fact in all cases I know of.[1]  I think
you vastly underestimate the cost to reviewers and authors of doing a
sufficiently good job to justify trusting the code at a significantly
higher level than currently.  (Authors will have to rework features
rejected for security of implementation reasons, and it would be quite
painful to have a whole feature rejected for security reasons).

 > The GNU ELPA, being enabled by default, should be treated the same.  But
 > because it's a network resource, we must use signatures to indicate
 > files in the GNU ELPA can be trusted, since we don't package the GNU
 > ELPA with Emacs itself.

No, what signatures are for is to authenticate the file, ie, to prove
that this is indeed a true GNU ELPA file and not a counterfeit.  Trust
will be allocated to *code received from* GNU ELPA on the basis of the
review policy and its quality of implementation, not simply because
it's signed.

 > In other words, we're building a web of trust around GNU Emacs because
 > we want to be able to install parts of it (hosted in the GNU ELPA)
 > optionally and over the network.  That's where "trust" matters.

Why do you think anybody in this thread doesn't understand that?

 > I never said it would be cheap or easy to do this.

True.  *I* *did* say that it would be expensive.  Very.  What worries
me is that you make statements like

 > But I think the FSF and GNU volunteers can handle the task, and I
 > firmly believe reviewing Emacs Lisp code is much easier than C,
 > C++, Perl, etc. code.

without offer any cost estimate (eg, # of commits per month, amount of
volunteer time per month, amount of time spent doing background checks
on volunteers, amount of time spent making sure that volunteers aren't
conspiring with Untrustworthy Authors, etc).  Nor do I see why
reviewing Emacs Lisp code is *much* easier.  No buffer overflows, it's
true, but magic execution (aka hooks) are all over the place.
Anything that reads a file using find-file (let alone `load') can
execute arbitrary Lisp, in fact, arbitrary executables using
call-process via hooks.

 > I'll volunteer my own time for these reviews, unless of course you
 > or others are opposed because it would make me a "security tzar" or
 > because it's felt I'm unqualified.

I don't think you're unqualified.  I do wonder why you think you're
*more* qualified than package authors.  AFAIK you're not currently a
GNU ELPA maintainer?

 > As I said, the alternative is the current "just trust whatever we
 > put online" model, which is certainly cheap to run.

And you're wrong about that.  People *do* just trust, by and large,
but that is *not* their only option, even under current conditions.
In fact, as far as I know many people are *currently* exercising a
different option by only trusting a few packages that are put online
at ELPA.

 > SJT> The obvious candidates for the latter are the authors.
 > 
 > No.

Yes.  I wrote "candidate", not "automatic pass".  *Many* authors (ie,
committers, since it has to be committed to GNU ELPA) are well-known.
True, it would be a bad idea to allow any random patch author to
certify their work, but if it has passed through the affected
package's community's review and been signed by a well-known package
author (ie, its maintainer), I can't see any claim that a review by an
unspecified "volunteer" who may not know anything about the package is
more trustworthy.  No?

 > SJT> If they are not security reviews, what's the point of reviewing at
 > SJT> all?  You just want signed commits, verifying that the commit was
 > SJT> actually received at the GNU ELPA.  AFAICS this can be done by a bot,
 > SJT> which checks the authors' signatures on the commits.
 > 
 > No.

No what?  That's a moderately complex piece of logic you're denying.

Steve

Footnotes: 
[1]  That could be because Emacs has a security cabal and they don't
discuss their reviews on this list.