Fwd: [bug #33138] .PARLLELSYNC enhancement with patch

[Tim Murphy]

---------- Forwarded message ----------
From: Tim Murphy <address@hidden>
Date: 25 April 2013 07:13
Subject: Re: [bug #33138] .PARLLELSYNC enhancement with patch
To: "Paul D. Smith" <address@hidden>

To be honest, I have done all this before with named semaphores including the "file that gets left over" problem and it's all solvable quite nicely.  You pass the build id in the environment which is, after all, what it's for.

It has been done on huger builds than most people using make ever imagine and contention doesn't appear to be a problem as the semaphore is held momentarily while writing output.

I'm not saying that this clever stuff is not a good idea but the reasons for doing it haven't proved to be valid in my experience.

On 24 April 2013 23:47, Paul Smith <address@hidden> wrote:

On Wed, 2013-04-24 at 22:55 +0100, Tim Murphy wrote:
> why not use a named semaphore wherever possible (windows and linux)
> and lock a file where not instead of trying to pass kernel object
> handles around (seems a bit nasty to me)?

Hi Tim; I think you're late to the party :-).  Let me summarize a lot of
discussion then we can use this as a reference for other questions.

Named semaphores on POSIX are kind of sucky.  They don't get
automatically cleaned up when the process exits, for one thing.  And my
suspicion is that they're not very portable across different variations
of UNIX especially older ones.

File locking, on the other hand, is very portable, very simple, very
fast, and any held locks are automatically released when the descriptor
is closed.  No muss, no fuss.

If you agree file locking is the right way to go on a POSIX system, then
we just have to decide what to lock.  It can be anything as long as all
children of a given top-level make can find it.  We could use a
temporary file, sure.  If we did that we'd have to pass around either
the name of the file or, more likely, use tmpfile() and send along the
file descriptor, handle close-on-exec, etc. like we do with the
jobserver pipe.  Which we could certainly do: we do it today with
jobserver.

However we already have a descriptor available to us: stdout.  If we use
that we don't have to pass around anything because our children are
already inheriting it and it's on a well-known FD.  We don't have to
worry about using "+" before sub-make rules like we do with jobserver.

There's one other advantage of using stdout: if a sub-make redirects its
output to a separate file, that magically starts a new "lock domain" and
that sub-make and its children don't contend with the other sub-makes
and their children.  This is not a big deal and if we didn't get it for
free, we'd never go to the effort of implementing it.  But it's nice.


Eli is working on the Windows port; I have no idea how he's decided to
implement this there yet.

--
You could help some brave and decent people to have access to uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/

--
You could help some brave and decent people to have access to uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/