Re: Automatically Handling Tools that Produce Many Outputs

[Olly Betts]

On 2007-11-27, Bob Friesenhahn <address@hidden> wrote:
> On Tue, 27 Nov 2007, Olly Betts wrote:
>> My experience is that multi-output rules often aren't protected at all,
>> and parallel make is unreliable on such projects.  This is becoming much
>> more of an issue, as multicore boxes are now mainstream.
>
> What relationship is there between parallel make and multicore? 

Parallel make is more beneficial on boxes with more cores.

> Parallel make has been in use long before most people had access to 
> multiple CPUs.  Even with a single CPU, processes block waiting for 
> I/O.

In my experience at least, a parallel make on a single CPU can run a bit
faster.  But if the makefiles don't handle it, it's not really a big
deal.

Such makefile bugs typically aren't reliably repeatable, and fixing them
properly is suprisingly complicated, so people often don't.  Just
rerunning the make command will often (by chance) avoid the bug and
there's always the easy workaround of running a non-parallel make when
the problematic rules need to be run.

With multiple cores, parallel make will generally run several times
faster.  So (as I said) lack of support for it becomes much more of an
issue.

> I definitely agree that all build paths need to be supported with a 
> make rule.  Produced objects which are a side-effect of other rules 
> cause problems for parallel builds.  Unfortunately, make itself is not 
> very helpful here since it evaluates just one target at a time. 

Indeed, which is why the solutions suggested in the automake manual are
(by necessity) rather complex.

Cheers,
    Olly