Re: How Do Parallel Builds Scale?

[NightStrike]

On Sat, Nov 17, 2012 at 7:30 AM, Bob Friesenhahn
<address@hidden> wrote:
> I suspect that this anaysis has been mentioned on the Automake list before
> but (if so) it is worth looking at again.  It seems that few packages
> benefit significantly from parallel builds.  Many packages use Automake, but
> they use it in a very inefficient way.  Here is a web page which shows the
> current pitiful state of parallel builds:
>
>   http://hubble.gforge.inria.fr/parallel-builds.html
>
> Given that even desktop and laptop computers support 4 CPU cores and that
> affordable server type hardware is available with as many as 64 CPU cores,
> it is useful if the build process of packages is redone so that it benefits
> from compilation on modern hardware.

http://mingw-w64.sf.net/ makes excessive use of massive parallelism.
In fact, we can build every single file in parallel, thanks to a
strong focus on non-recursive make.  At one point in our build
process, we have to build several thousand libX.a files from
corresponding X.def files.  This can be parallelized down to 1 job per
library.