Re: [bug #40225] Deterministic output ordering

[Frank Heckenbach]

Paul D. Smith wrote:

> I don't think Frank's suggestion below will work (or maybe I misunderstood
> it).  It's not enough to remember the order in which target recipes were
> started, because in parallel builds the order of starting targets can be
> different.
>
> [...]
>
> What we'd need to do is remember the order in which make _would_ have built
> the targets, if running in a serial way.  In order for that to work, we'd need
> to keep an ordered list of the targets we visit as we walk the dependency
> graph, and we'd need to convince ourselves that even with parallelism enabled
> we will walk the graph the same way without skipping any nodes (that is, we
> need to be sure that we won't jump to another branch of the graph because we
> know that this branch is blocked by a target currently being built).
> 
> These changes would need to be made above the job layer, in remake.c which is
> where the dependency graph is walked.  Frank is correct that we'd need to
> separate the output dump function out and hoist it up a layer, so that it
> could be done in the proper order.

I see. In this case (I'd been waiting for a definitive statement
from you) I agree it's quite complicated and the better way to
achieve the goal may be to implement "tree structured log files" as
was suggested before (so external tools can concatenate them in a
deterministic order, e.g. alphabetically, or parse them file by file
in the first place).

As I wrote before, this might not be so hard to implement by using
the output-sync infrastructure, just changing it so the log files
are not created as temporary files, but as actual log files with
names based on the target, and later not dumped and deleted. Of
course, there's some devils in the details, like how to reliably
name those log files, making sure they're unique, whether to combine
or separate stdout and stderr, where to put make's output messages
that don't apply to a particular target etc.