Re: GOP-PROP 5: build system output (update)

[Phil Holmes]

----- Original Message ----- 
From: "Graham Percival" <address@hidden>
To: "Phil Holmes" <address@hidden>
Cc: <address@hidden>
Sent: Friday, July 15, 2011 4:43 PM
Subject: Re: GOP-PROP 5: build system output (update)


> On Fri, Jul 15, 2011 at 02:20:51PM +0100, Phil Holmes wrote:
> > ----- Original Message ----- From: "Graham Percival"
> > >   * All output will be saved to various log files. (including
> > >     output from make(1))
> > >   * We will still display the output of make(1) on the console.
> >
> > I read these as mutually exclusive.  Are you saying we'll send the
> > output of make both to a console and a log file?  Why?
>
> Yes.  (well, "proposing" rather than "saying that we will")
>
> - package maintainers (evidently) want to see the normal output of
>  make(1).  Also, it anybody doesn't want to see that, they can
>  trivially avoid it by doing make --silent.
> - we'll still have tons of output, and some terminals only save
>  1000 lines (for example).  If something goes wrong and I want
>  to see exactly what commands were run, it'll be much easier to
>  look at those commands if they were dumped verbatim into a
>  test file.
>
> I'm not certain if it's possible to cause make(1) to automatically
> put its output into a logfile in addition to the console.  If not,
> or if it looks really finicky to do this, then I'm quite willing
> to withdraw this aspect of the proposal.  But if we're discussing
> our ideal for the build system, mine would include this.

> From my reading of the manual, there is no option to redirect make output to 
a logfile.  I always do this with a command line redirect.  AFAICS the only 
other way to do this would be to wrap make in another program that captures 
its output to log and console - Kili wrote something like this.  However, 
I'm personally rather loathe to do such wrapping.

> > >   * No other output will be displayed on the console, with one
> > >     exception: if a build fails, we might display some
> > >     portion(s) of log file(s) which give useful clues about the
> > >     reason for the failure.
> >
> > I think there should be an option to turn it all back on if you want
> > - a sort of inverse of QUIET_BUILD.  We should also get rid of the
> > QUIET_BUILD variable completely.
>
> Agreed.  Maybe using the V=1 thing that Jan was talking about?

That sort of thing.  I'd propose a variable called VERBOSE that would have 
to be set to get the output on screen.

> > >   * Both stderr and stdout will be saved in *.log. The order of
> > >     lines from these streams must be preserved.
> >
> > I do not believe this is possible, given the buffered nature of
> > stdout.  See the font of all Unix knowledge, Wikipedia, when
> > discussing sending both streams to the same file:
> >
> > "Messages appear in the same order as the program writes them,
> > unless buffering is involved. (For example, a common situation is
> > when the standard error stream is unbuffered but the standard output
> > stream is line-buffered; in this case, text written to standard
> > error later may appear on the terminal earlier, if the standard
> > output stream's buffer is not yet full.)"
>
> Hmm.  I've heard people talking about this, but I can't recall
> ever running into this problem in lilypond development.  That
> said, I almost always do multi-cpu builds, so I'm never surprised
> when I see error messages apparently out of order -- but the
> errors always make sense when I try a second make(1) call without
> multi-threading.
>
> Can we change the buffering on streams?  I wouldn't be surprised
> if we could do that.  Or maybe this is an argument in favor of
> always using stderr, since that (apparently) guarantees unbuffered
> output?
>
> At the moment, I'm thinking that we should acknowledge this as a
> theoretical possibility, leave it in the policy as an "intended
> behavior", and then go on with life.  If we have a bug report
> about the actual behavior not following the intended behavior,
> then we'll look at the technical issues in that bug, and if the
> technical issues are unsurmountable, we can modify the policy.

Agreed.  I honestly don't think the slight possiblity of out-of-order 
messages should stop this.

> > >   * Ideally, a failing build should provide hints about the
> > >     reason why it failed, or at least hints about which log
> > >     file(s) to examine.
> >
> > Seems a repeat of bullet point 3?
>
> I think there's miscommunication here.  I see bullet point 3 as "
> Logfiles from calling lilypond... all other logfiles will go
> in..."
> That bullet point is talking about informing the user of *which*
> logfile to examine (we'll have a lot of logfiles, so guesswork
> wouldn't be fun!), and possibly even output to console a portion
> of the relevant logfile.
>
> How could I clarify the language in those bullet points, or did
> you mean a different bullet point?

I read the 2 bullets about indicating log files to examine as being similar. 
You're right - they're somewhat different.

--
Phil Holmes