Re: parallelism: job slots vs. levels

[Howard Chu]

Dan Jacobson wrote:
> Anyways,
> $ make x& make y& wait
> cannot always be rewritten with -j.
> $ make -j[whatever number] x y
> will act differently except for special cases of x and y;
> probably when both x and y have no dependencies.
>
> Anyways, with -j examples added to the manual, we would get on the
> right track about how to use -j.

I'm not quite sure what the point of all this is. I guess the texinfo 
docs for parallel execution may be somewhat lacking. Here's a summary of 
Things You Must Know.

In a typical Makefile with target and dependencies:

foo: bar baz
        cmds


the Make syntax specifies that all of the dependencies are of equal 
precedence. That means, bar and baz can be made in any order and the 
result must still be valid. If there is really a dependency such that 
bar must be created before baz, then the above rule is incorrect. (Or at 
least, insufficient.) It should be written instead as

baz: bar
        cmds

foo: baz
        cmds

In serial make you can get away with the incorrect syntax, because make 
only fires off one command at a time and it typically runs dependencies 
in left-to-right order. But that's just an idiosyncracy of the 
particular implementation. You could run across a "make" implementation 
that always processes dependencies right to left. If your Makefile 
follows correct syntax, it will still work. If it depends on 
left-to-right serial execution, it breaks, as well it should.

In parallel make, all the dependencies of a target can be processed 
simultaneously. As such, it is imperative that you *explicitly* define 
all dependency relationships in your Makefile. You can't rely on the 
*implicit* relationship that says that left-most dependencies will be 
built before right-most.

If in your example of "make x& make y& wait" "x" and "y" are completely 
indepedent, then the result will be correct, as would "make -j x y". But 
if you're relying on some implicit behavior to complete "x" before 
starting "y" then your Makefile is broken.

The point is that in a correctly written Makefile with all dependency 
relationships explicitly spelled out, there will be no difference 
whether make is run serially or in parallel. If there is any difference, 
then the Makefile rules didn't correctly capture all of the real 
dependencies of the project.
--
  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support