RFC: docker for CI

[Han-Wen Nienhuys]

Proposal: rather than using the patchy scripts for validating
LilyPond, we use docker images.

General idea
============

There is a script ("driver") that drives docker running on a dedicated
build machine ("host").

There are several images:

* The base dev image.

The base image is based on some stripped Linux distribution, with all
the devtools necessary for compiling LilyPond. In addition, it
contains a copy of ccache, and a git clone of the LilyPond sourcecode

* The base release image for a specific git commit.

The procedure to build it is as follows:

  * take the base dev image
  * fetch the git commit
  * runs (make ; make test-baseline)
  * runs (make dist-clean)

This saves the result as a docker image. The Docker image now contains
a clean lilypond tree, the C++ compilation results (in ccache), and a
test baseline.

The base release image is made at official LilyPond releases, or at
any release that has a new graphical regtest result


CI: build binary
================

Given a proposed change (as git commit):

 * take base release image
 * run (make; make doc) >& log-file

On success, the driver saves the result as a docker image, tagged with the
commit sha1.

On failure, the driver uploads the last bit of the log-file to our code
review system.


CI: regtest
===========

Given a proposed change (as git commit)

  * take CI build image
  * run (make check >& log-file)
  * use a headless browser to take a image snapshot of the top of regtest
result page.


On success, the driver uploads the image snapshot to code review.

On failure, the driver uploads the last bit of the log-file to code review.


Considerations
==============

* Because the build happens inside a container, we can test multiple
  builds. We could build against guile 1.8 and 2.2 at the same time,
  for example

* Because the "build binary" step reuses CCache results, it can
  complete quickly.

* The regtest continues to be expensive to compute. In the future, I
  hope it would not need a human to kick it off or post results back
  into review, but likely, it should require a manual step in the
  review process to kick off, eg. in Gerrit "Run-Regtest" +1 vote.

* For security, the host should use https://github.com/google/gvisor
  to avoid being hacked by malicious code in proposed changes.

--
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen