GOP-PROP 5: build system output (probable decision)

[Graham Percival]

Not much response from the previous GOP-PROP 5 (update); I'm not
certain if "silence is a form of consent" [1] in this context.

[1] true story from a friend's lifeguard training.  If somebody if
choking but declines any help, the lifeguard (in Canada, at least)
is legally obliged to watch the person collapse.  Once they've
lost consciousness from lack of oxygen, the lifeguard should ask
"sir, do you require assistance", and when there is no response
from the unconscious body, the lifeguard is allowed to provide
medical help.


http://lilypond.org/~graham/gop/gop_5.html

(proposal written by Phil Holmes, modified by Graham)


** Proposal summary

When you run make or make doc,

    * All output will be saved to various log files, with the
      exception of output directly from make(1).
    * By default, no other output will be displayed on the
      console, with one exception: if a build fails, we might
      display some portion(s) of log file(s) which give useful
      clues about the reason for the failure.

      The user may optionally request additional output to be
printed; this is controlled with the VERBOSE=x flag. In such
cases, all output will still be written to log files; the console
output is strictly additional to the log files.

    * Logfiles from calling lilypond (as part of lilypond-book)
      will go in the relevant
      ‘build/out/lybook-db/12/lily-123456.log’ file. All other
      logfiles will go in the ‘build/logfiles/’ directory.
    * Both stderr and stdout will be saved in *.log. The order of
      lines from these streams should be preserved.
    * There will be no additional “progress messages” during the
      build process. If you run make --silent, a non-failing build
      should print absolutely nothing to the screen.
    * Ideally, a failing build should provide hints about the
      reason why it failed, or at least hints about which log
      file(s) to examine. 

** Rationale

Before any of the current work on reducing output from make, the
result of a "make doc" was over 500,000 lines of text. Finding
errors or warnings in that volume of output is always going to be
hard. The prime reason for the output being so verbose is that all
the processes that run as a result of the call to make echo their
output to the screen, often in verbose mode. Lilypond itself
produces around 370,000 lines of output as a result of
lilypond-book building all the snippets.

Much of this output can be redirected to logfiles and so the
impossible-to-read clutter on the screen is cut out and could be
referred to later. However, there is a danger in this approach,
that vital error messages can also be lost, thus preventing the
cause of the failure of a make being found. We therefore need to
be exceptionally careful to move cautiously, include plenty of
tests, and give time for people to experiment/find problems in
each stage before proceeding to the next stage.

A variable, QUIET_BUILD, can be set and this will reduce the
clutter but not eliminate it. (see
http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables
) This variable currently does things like adding a -q flag to the
TEXI2PDF call ("quiet") and getting rid of the –verbose flag in
some calls to LilyPond. However, it could be used more widely, as
proposed below.


** Implementation notes

none yet 


Cheers,
- Graham