Re: Make and Clusters

[Martin d'Anjou]

Hi David,

I explored this problem a while ago. I captured my musings in a blog:
https://ninjaverification.wordpress.com/2008/10/21/overriding-gnu-make-shell-variable-for-massive-parallel-make/

In the end the idea is to have make call out to a job scheduler as part
of the target recipe. This is the method I use and it works really well.

One problem I still have though is that if one of the parallel jobs
aborts and compromises the whole build, make will wait for all other
in-flight jobs to finish before itself returning. At the time I had
manually patched my local copy of GNU Make, but the patch no longer
applies and I have not gotten back to it. Here is a reference in case
you find it useful:
http://savannah.gnu.org/bugs/?41781

Best,
Martin

On 17-10-10 09:53 AM, David Delahaye wrote:
> Hello,
>
> I wonder how the make command is compatible with clusters. I know the
> "-j" option, which allows us to run several jobs simultaneously and
> uses the several cores of a given processor. But if we have several
> nodes of computation of a cluster, is the command "make -j" able to
> run over these nodes (transparently for the user)? I know there are
> specific commands to run jobs on a cluster, I would like to know how
> "make -j" is compatible with these commands. Thanks in advance for
> your help.
>
> Best regards,
>
> David Delahaye.
>