bug-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Make run in parallel mode with output redirected to a regular file c


From: Stefano Lattarini
Subject: Re: Make run in parallel mode with output redirected to a regular file can randomly drop output lines
Date: Sun, 26 May 2013 11:14:02 +0200

On 05/26/2013 05:18 AM, Frank Heckenbach wrote:
> (Sorry, I can't reply properly; I'm not subscribed, just saw the
> message on the web archive.)
> 
> Stefano Lattarini wrote:
> 
>> Make run in parallel mode with output redirected to a regular file
>> can randomly drop output lines
> 
> Yet another reason to use the new --output-sync feature. :)
> 
Definitely, but only when redirecting the output to a regular file.

> I hadn't actually thought about this particular problem when I got
> interested in the feature, but it makes sense. Perhaps I had
> occasionally lost some messages before and never noticed ...
> 
>> The issue is present both in all of make 3.81, make 3.82 and make
>> built from latest Git.  Here is a script that demonstrates it:
>>
>> [...]
>>
>> and the following suggests it might not be easy to fix:
>>
>>   <http://lists.gnu.org/archive/html/automake-patches/2009-03/msg00073.html>
> 
> As Ralf Wildenhues explains in that mail, it's not really a make
> problem, but the behaviour of POSIX files, also "Using O_APPEND
> avoids this race", i.e. in your demo script, using ">>" (and
> clearing the target file before) would also fix the problem.
>
I know, I just refactored the whole Automake testsuite a few days
ago to fix the issue in the same way you suggested ;-)
The issue is not at all difficult to work around, *once* you know
it's there and what its causes are.

>> But it's worth nothing that the issue is not present with FreeBSD make (as
>> offered by Debian package freebsd-buildutils 9.0-11); maybe the sources of
>> that package might suggest how to obtain a fix after all?
> 
> I suppose it does something similar to our output-sync, i.e.
> directing the output from different jobs to different temp files and
> dumping it to the original stdout/stderr synchronized.
>
Yes, I think you are right.

> Or do you have something else in mind that make should (and could) do?
> 
Actually no; maybe it is *possible* to play some tricks with the
inherited file descriptors in order to mitigate or fix the issue,
but I haven't given it any real thought.  Nor I will anytime soon
in all likelihood.  I just wanted to report this IMO subtle issue
with the hope of making awareness of it more widespread, since
AFAIK even very experienced developers had been bitten by it.

An now that I think about it, maybe the sanest "fix" would be just
documenting the issue in the manual?

Thanks, and best regards,
  Stefano



reply via email to

[Prev in Thread] Current Thread [Next in Thread]