Re: [bug #33138] .PARLLELSYNC enhancement with patch

[Paul Smith]

On Thu, 2013-04-18 at 19:09 +0200, Frank Heckenbach wrote:
> This mechanism was unaffected by my output-sync patch, and I
> expected your change broke it.

I was reading your email with interest, waiting for the punch-line, but
then after all that description you just said that the change broke it,
with no explanation :-).  My question is, what about the new behavior
does not work for you?

The change I made was to have all the jobs in a target recipe write
(appending of course) to the same temp file, then after the last job in
the recipe is completed the output for the entire recipe is generated.
So for example, if you have:

  foo bar:
          : $@ one
          : $@ two
          : $@ three

and you run "make -j -O", the way you had it the output from each job
would be mixed something like this (the actual result order will be
slightly random):

  foo one
  bar one
  foo two
  bar two
  bar three
  foo three

with my change you still get the same results but only after all jobs
are complete, and they will be collected:

  foo one
  foo two
  foo three
  bar one
  bar two
  bar three

> So I'd plead to revert this bit (since one can still use .ONESHELL
> if wanted). Or we could add another mode like "--output-sync=job".
> Shouldn't be too hard now (if you like, I can implement it).

I'd prefer to not add another option here unless there's a compelling
case for it, even though you're correct that we have the flexibility
now.

> > I think we're
> > doing too much here.  I would have left it alone but I think some tools
> > that parse make results expect to see that output to help them figure
> > out what's going on.
> 
> As I described in https://savannah.gnu.org/bugs/?33138#comment3, the
> problem is how to interpret messages that contain file names when
> different jobs run in different subdirectories.

I do get it.  I just think we might need to do more drastic surgery on
this to REALLY get it right.  Probably we'll want to allow the user to
have more control over it, as well.  Maybe a similar flag that lets you
choose whether to trace on a per-target or per-make basis.  And, if you
choose per-target you don't need to ALSO generate output per-make.

We do need to be careful about tracing non-target output, for example
output generated by $(info ...), $(warning ...), and $(error ...).

> > I guess this points out a potential issue with the current UNIX
> > implementation as well: if you had some utility you were running as part
> > of your build that was also trying to lock stdout, then it would
> > interfere with make.
> 
> I don't think so, since its stdout is now our temp file which it may
> lock as it pleases. (In fact, that's just what recursive makes do
> with --output-sync=make.)

True.  And if they were trying to use stdout like we are, to lock
between different recipes, they just can't use -O at all since by
definition each target will be writing to a different file.

> PS: In assign_child_tempfiles(), the following declarations are no more
> needed and can be removed:
> 
>   FILE *outstrm = NULL, *errstrm = NULL;
>   const char *suppressed = "output-sync suppressed: ";
>   char *failmode = NULL;

Yeah, I have a commit in my local repo that fixes this.

Thanks!