Re: [RFC]serialize the output of parallel make?

[Edward Welbourne]

>>> The shell wrapper buffers the recipe output and then grabs a semaphore
>>> before writing the output to it's stdout..  if another recipe has
>>> completed and is in the process of outputting to the stdout then it
>>> has to wait a few microseconds.
>> The use of semaphore may impair performance.
> And "serialzation" you mean is not the same as I mean.

> I believe Paul and Edward fully understand what I mean.

The approach described above is - as far as I can tell - exactly what
my kludge does, except done cleanly.  In particular, my kludge has the
same semaphore problem (it uses flock) and I'm quite sure this doesn't
impair performance - because each recipe does the thing that actually
matters (compiles the code) before getting stuck waiting for an
opportunity to deliver its output (synchronously) to the common pool.
So the only cost is that the process that ran the recipe is sitting
idle for a while after it finishes its work, before it reports its
results.  This might lose you some performance if your -j count is too
low, but I always use a generous -j limited by a suitable -l in any
case (i.e. I let make run as many processes as it finds useful, so
long as it doesn't take the load above a limit I set).

So I think Tim Murphy understands perfectly well what we mean by
serialization - ensuring the output from each command comes out
cleanly, separate from the output of any other command - and I'm
suddenly interested in finding out more about talon ;-)

        Eddy.