[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallelism: job slots vs. levels

From: Howard Chu
Subject: Re: parallelism: job slots vs. levels
Date: Tue, 31 Aug 2004 22:41:03 -0700
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a2) Gecko/20040714

Dan Jacobson wrote:
$ make x& make y& wait
cannot always be rewritten with -j.
$ make -j[whatever number] x y
will act differently except for special cases of x and y;
probably when both x and y have no dependencies.

Anyways, with -j examples added to the manual, we would get on the
right track about how to use -j.

I'm not quite sure what the point of all this is. I guess the texinfo docs for parallel execution may be somewhat lacking. Here's a summary of Things You Must Know.

In a typical Makefile with target and dependencies:

foo: bar baz

the Make syntax specifies that all of the dependencies are of equal precedence. That means, bar and baz can be made in any order and the result must still be valid. If there is really a dependency such that bar must be created before baz, then the above rule is incorrect. (Or at least, insufficient.) It should be written instead as

baz: bar

foo: baz

In serial make you can get away with the incorrect syntax, because make only fires off one command at a time and it typically runs dependencies in left-to-right order. But that's just an idiosyncracy of the particular implementation. You could run across a "make" implementation that always processes dependencies right to left. If your Makefile follows correct syntax, it will still work. If it depends on left-to-right serial execution, it breaks, as well it should.

In parallel make, all the dependencies of a target can be processed simultaneously. As such, it is imperative that you *explicitly* define all dependency relationships in your Makefile. You can't rely on the *implicit* relationship that says that left-most dependencies will be built before right-most.

If in your example of "make x& make y& wait" "x" and "y" are completely indepedent, then the result will be correct, as would "make -j x y". But if you're relying on some implicit behavior to complete "x" before starting "y" then your Makefile is broken.

The point is that in a correctly written Makefile with all dependency relationships explicitly spelled out, there will be no difference whether make is run serially or in parallel. If there is any difference, then the Makefile rules didn't correctly capture all of the real dependencies of the project.
  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

reply via email to

[Prev in Thread] Current Thread [Next in Thread]