Re: GOP-PROP 5: build system output (update)

[Phil Holmes]

----- Original Message ----- 
From: "Graham Percival" <address@hidden>
To: <address@hidden>
Sent: Thursday, July 14, 2011 5:09 PM
Subject: GOP-PROP 5: build system output (update)


> Update on this; I'm not ready to call it a "probable decision"
> yet.
> http://lilypond.org/~graham/gop/gop_5.html
>
> (proposal written by Phil Holmes, modified by Graham)
>
>
> ** Proposal summary
>
> When you run make or make doc,
>
>    * All output will be saved to various log files. (including
>      output from make(1))
>    * We will still display the output of make(1) on the console.

I read these as mutually exclusive.  Are you saying we'll send the output of 
make both to a console and a log file?  Why?

>    * No other output will be displayed on the console, with one
>      exception: if a build fails, we might display some
>      portion(s) of log file(s) which give useful clues about the
>      reason for the failure.

I think there should be an option to turn it all back on if you want - a 
sort of inverse of QUIET_BUILD.  We should also get rid of the QUIET_BUILD 
variable completely.

>    * Logfiles from calling lilypond (as part of lilypond-book)
>      will go in the relevant
>      ‘build/out/lybook-db/12/lily-123456.log’ file. All other
>      logfiles will go in the ‘build/logfiles/’ directory.
>    * Both stderr and stdout will be saved in *.log. The order of
>      lines from these streams must be preserved.

I do not believe this is possible, given the buffered nature of stdout.  See 
the font of all Unix knowledge, Wikipedia, when discussing sending both 
streams to the same file:

"Messages appear in the same order as the program writes them, unless 
buffering is involved. (For example, a common situation is when the standard 
error stream is unbuffered but the standard output stream is line-buffered; 
in this case, text written to standard error later may appear on the 
terminal earlier, if the standard output stream's buffer is not yet full.)"

>    * There will be no additional “progress messages” during the
>      build process. If you run make --silent, a non-failing build
>      should print absolutely nothing to the screen.
>    * Ideally, a failing build should provide hints about the
>      reason why it failed, or at least hints about which log
>      file(s) to examine.

Seems a repeat of bullet point 3?

> ** Rationale
>
> Before any of the current work on reducing output from make, the
> result of a "make doc" was over 500,000 lines of text. Finding
> errors or warnings in that volume of output is always going to be
> hard. The prime reason for the output being so verbose is that all
> the processes that run as a result of the call to make echo their
> output to the screen, often in verbose mode. Lilypond itself
> produces around 370,000 lines of output as a result of
> lilypond-book building all the snippets.
>
> Much of this output can be redirected to logfiles and so the
> impossible-to-read clutter on the screen is cut out and could be
> referred to later. However, there is a danger in this approach,
> that vital error messages can also be lost, thus preventing the
> cause of the failure of a make being found. We therefore need to
> be exceptionally careful to move cautiously, include plenty of
> tests, and give time for people to experiment/find problems in
> each stage before proceeding to the next stage.
>
> A variable, QUIET_BUILD, can be set and this will reduce the
> clutter but not eliminate it. (see
> http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables
> ) This variable currently does things like adding a -q flag to the
> TEXI2PDF call ("quiet") and getting rid of the –verbose flag in
> some calls to LilyPond. However, it could be used more widely, as
> proposed below.
>
>
> ** Implementation notes
>
> none yet
>
>
> Cheers,
> - Graham
>
> _______________________________________________
> lilypond-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/lilypond-devel


--
Phil Holmes