Re: [bug #33138] .PARLLELSYNC enhancement with patch

[Frank Heckenbach]

Paul Smith wrote:

> *sigh*  If it weren't for the enter/leave messaging, the output-sync
> feature would have been quite straightforward! :-/ :-).

I'm afraid so. But I think we're almost there now.

> On Mon, 2013-09-16 at 12:18 +0200, Frank Heckenbach wrote:
> 
> > 1.
> > 
> > Twice "Entering", once "Leaving". I think there's an output_close()
> > missing before reexec (which might also lose other messages in other
> > situations):
> 
> I didn't fix it this way.  Instead I used the existing MAKE_RESTART
> environment variable to communicate from the current make to the
> restarted make that the enter message was already printed (if it was) so
> it wouldn't be printed again.  This ensures we don't get a stream of
> enter/leave statements as we re-exec.
> 
> This works now (in my repo).

This seems even better. However, you still might have to call at
least output_dump() before re-exec, especially if you sync the
makefile parsing as discussed below.

> > 2.
>
> I'm not sure I agree with that assessment.  I spent a bit of time
> considering how "recurse" should work, and it's very different than
> line/target.  This is because in "recurse" mode we know that our parent
> make has redirected all our output to a temp file, so it won't get
> dumped until the entire make is complete, while in the other modes we do
> not redirect output from recursive command lines so any output generated
> by them goes directly to stdout.
> 
> As a result, when it comes to enter/leave the "recurse" mode is actually
> the same as non-sync mode (or even non-parallel mode): we only need to
> write enter/leave at the beginning and end of the entire make process,
> not around every recipe.  The individual recipes are still sync'd, of
> course, but we don't need enter/leave around each one because they all
> get written to a single temp file and that temp file will have a global
> enter/leave when it is dumped.

Thanks for the explanation, that's what I was missing. Now I
understand your rationale for doing it the way you did.

> > 3.
>
> Hm.  I suppose you mean that this will "bundle" the output from makefile
> parsing along with the output of the first synchronized output (line or
> target).  I don't think that will work well, since the makefile output
> is not synchronized the enter message could be a long way from the first
> target or line output.
>
> I think synchronizing the makefile parsing will yield better results.

I think you're right, that's the way to go.

> > 4.
> > 
> > Here, with -Otarget or -Oline there's no Enter/Leave messages if
> > there's no other output. Not sure if you consider this an actual
> > problem. The solution seems to be to redirect stderr (only) before
> > $(shell) executions just like we do for recipe jobs.
> 
> Hm.  This is pretty contrived.  I have a hard time imagining a real
> makefile wanting to do this for a good reason.  However, it does seem
> that the solution may be simple enough.

I also doubt someone would do it intentionally, but of course, there
could be a $(shell) command that writes some error or warning to
stderr (perhaps only under certain circumstances, not expected by
the original author). Then again, this situation may be so rare that
it might not warrant extra effort.

> > 7.
> > 
> > job.c:1258: OUTPUT_UNSET();
> > 
> > Just wondering, is this necessary? (It's before OUTPUT_SET().)
> 
> It is, because if you look above you'll see there's a label
> next_command: and later in the function we goto that label, perhaps with
> OUTPUT_SET().
>
> PS. I know this use of goto is nightmarish: I didn't write this code :-)

Oh yes, sorry, I missed this. (I'm not trained to look for gotos. ;-)