[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Parallel make

From: Tim Murphy
Subject: Re: Parallel make
Date: Wed, 29 Apr 2015 21:46:33 +0100

Without wanting to turn this into a commercial/advert you might want to consider trying the Electric Cloud Huddle beta since it works with multiple machines in a convenient way and deals with the problems of getting correct parallel builds. It is also free for now at least.

Sorry for that :)

On 29 Apr 2015 11:20 pm, "Paul Smith" <address@hidden> wrote:
On Wed, 2015-04-29 at 13:50 -0600, Ryan P. Steele wrote:
> The multithreaded version of make (-j#) is wonderful, and we have made
> great use of it.  Because we're dealing with some very large code,
> however, it would be great to be able to parallelize compilation over
> multiple machines.  I can't seem to find any option for doing this,
> though.  Does such functionality exist?  It would appear to be a
> fairly straightforward extension of multithreading, especially on a
> network file system, but thus far, we haven't been able to make it
> work.  Any help would be appreciated.

Just to be clear, GNU make is not multithreaded.  It simply spawns
multiple processes and lets them run in parallel.

GNU make has no built-in capability to use multiple machines:
conceptually it may be a straightforward extension but the effort needed
to communicate between multiple systems over a network, send and receive
results reliably, kill jobs when someone stops the main make process
with ^C, etc. is FAR out of GNU make's current wheelhouse.

You can get a very cheap implementation, if you're willing to live with
many prerequisite configuration requirements such as a networked
filesystem, SSH access that doesn't require a password, etc. by writing
your own script to forward the job via SSH and setting the GNU make
SHELL variable to point to your script.

Luckily, if you are building C or C++ code someone has already done all
the necessary work for you.  I recommend you investigate the distcc
package: https://code.google.com/p/distcc/


Bug-make mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]