Re: possible solution for -Otarget recurse (was: Re: Some serious issues with the new -O option)

[Paul Smith]

On Fri, 2013-05-03 at 21:15 +0300, Eli Zaretskii wrote:
> > This will ensure that output from lines before the recursive make will
> > be shown before the targets in the recursive make.  It's not 100%
> > identical but I can't see any way to do better.
> 
> Why isn't it identical?

It's not identical in two ways: first it's not identical to -Otarget
because the output before and after the recursion are not shown in a
continuous block.  In:

  all:
          @echo start
          $(MAKE) -C subdir
          @echo end

the "start" and "end" will have other stuff (not just the other targets
in that sub-make, but ANY other targets that happen to finish during
that time) between them.  Obviously that's kind of the point;
nevertheless it's a difference.

Second, there's a difference from some ideal solution if the recursive
line generates output; suppose instead of the above you wrote it like
this:

  all: ; @echo one ; echo two ; $(MAKE) -C subdir ; echo end

Since the output of the entire recursive line is not synchronized but
just left to go to stdout, the output generated by the "echo" commands
can (a) interrupt other synchronized output, (b) have other output
interrupt it.

But, it will still be shown in order! :-)

> > Consider every target in the entirety of build, including all submakes
> > and all their targets as well, as one long list of targets (for example
> > the results of a serial build).  The fraction of those targets that are
> > invoking sub-makes will be, at most, very small.
> 
> That depends.  I think you will find a different fraction in Emacs,
> especially when it bootstraps itself (which involves byte-compiling
> all the Lisp files in a 3-level deep recursion, the last of which
> passes to Make a very long list of files which all can be compiled in
> parallel; see lisp/Makefile.in in Emacs).

Sure, but that's just one invocation of recursive make then that one
recursive make builds all those targets.  So that counts as one
recursive make target and lots of individual bytecompile targets.

Of course it's possible to write your makefile such that you run a
separate recursive make command for each target, but that seems pretty
wasteful of resources.

> > -Otarget wants to collect the output of each individual target, then
> > show the output for each target as soon as that target finishes.  That's
> > what users (should) expect when using this option.
> > 
> > In the case of recursive make targets, this presents an unsolveable
> > contradiction: how can you both show the output of EVERY target
> > (including the targets run by the submake) as soon as it completes, and
> > still not show the output of any target (including the targets that
> > invoke a submake) before it's complete?  You can't do both!
> 
> But I don't think there's a requirement to avoid showing incomplete
> output.  The only requirement is not to mix output from two or more
> parallel jobs, that's all.

That's the lowest-level requirement.  But if that were the ONLY
requirement we'd simply implement -Ojob and we'd be done.  That's all we
need to ensure that two or more parallel job outputs are not mixed:
-Ojob says that the output of each line of a recipe will be printed
without any interruption from any other lines running in parallel.

The -Otarget option makes a stronger statement: it says that the output
from all of the lines of a recipe will be printed together without any
interruption from any other recipe.  In the case of targets that
generate output AND invoke recursive make, we can't achieve this
stronger promise, at least not completely.