bug-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug #33138] .PARLLELSYNC enhancement with patch


From: Paul Smith
Subject: Re: [bug #33138] .PARLLELSYNC enhancement with patch
Date: Tue, 16 Apr 2013 12:56:21 -0400

On Tue, 2013-04-16 at 19:20 +0300, Eli Zaretskii wrote:
> > From: Paul Smith <address@hidden>
> > Cc: address@hidden, address@hidden, address@hidden
> > Date: Tue, 16 Apr 2013 10:44:39 -0400
> > 
> > On Tue, 2013-04-16 at 16:43 +0300, Eli Zaretskii wrote:
> > > > I'm not sure what the semantics of tmpfile() are on Windows.
> > > 
> > > The file is automatically deleted when closed.  But the documentation
> > > doesn't say what happens if it is open on more than one descriptor, or
> > > what happens if the original descriptor is dup'ed.  I will need to
> > > test that, and perhaps provide a work-around.
> > 
> > It might be that we have to allow use of a file handle on Windows,
> > rather than a descriptor.
> 
> That doesn't matter, really.  One can get one from the other on
> Windows.

Ah interesting.  In UNIX you can get _A_ file handle back from a file
descriptor (using fdopen()), but it's not guaranteed to be the SAME file
handle you had originally.  That is, if you run:

   FILE* f1 = fopen(...);
   int fd = fileno(f1);
   FILE* f2 = fdopen(fd, ...);
   fclose(f2);

you don't get back f2 == f1.  And although fd will be closed here, I'm
pretty sure not all the resources associated with f1 are freed, which is
a resource leak that will eventually lead to running out of file
handles.

> > The descriptor-based mutex has the very slight advantage over a
> > system-wide mutex in that if a sub-make's output is redirected it now
> > has its own lock domain.
> 
> I didn't mean a system-wide mutex, I meant a process-wide mutex.  Will
> this be OK?

I don't think so: especially now that we support full jobserver
capabilities in Windows we can have recursive make invocations all
running jobs in parallel, and we'd want them to synchronize their output
across multiple processes.  If we were only concerned about a single
process we really wouldn't need even mutexes since make is
single-threaded.

On the other hand I guess "system-wide" is not really right either.
Ideally what we want is a mutex shared between all the recursive make
instances and the root make, but we would still want multiple completely
independent make instances to not interfere with each other.

I guess this points out a potential issue with the current UNIX
implementation as well: if you had some utility you were running as part
of your build that was also trying to lock stdout, then it would
interfere with make.  That seems unlikely, but maybe to avoid this and
to deal with the potential issue of locking not supported on non-files,
we should just create a temporary file descriptor and pass it around,
like we do with the jobserver FDs.

> E.g., Make sees that both are connected to the same device and
> redirects them to the same file, but then the job redirects stderr to
> a separate file using shell redirection ("2> foo").  Or vice versa.

Sure... but I don't see the problem.  Maybe I've lost the thread.  When
the command starts both stdout and stdin are writing to the same
destination.  If the command does nothing special then the output will
be a combination of stdout and stderr in the order in which the command
generated them, which is good.  If the command script changes one or
both of stdin/stdout, then they won't be cojoined anymore, yes, but
that's what the user asked for...?




reply via email to

[Prev in Thread] Current Thread [Next in Thread]