Some "fun" last weekend

[Paul Smith]

So after the changes I made to allow exporting environment variables to
subshells, I wanted to work on the issues we have with jobserver pipes
getting confused when you run make in unexpected ways such as inside a
shell function or in a recipe line which is not considered a submake.

There are at least two Savannah bugs about this very subtle and
annoying issue.

So the first thing I tried to do was to add another setting of the
--jobserver-auth variable assignment to MAKEFLAGS of the child
environment if we're not in the "submake context".  So it would look
like this:

  MAKEFLAGS= --jobserver-auth=2,3 --jobserver-auth=-2,-2

where the "-2,-2" was there to override the original, and this would be
added only when starting a process that was not a recursive make.

This actually worked pretty well, but it's got some pretty frustrating
quirks (I think I alluded to these in the Savannah bug).  Mainly,
because we modify the environment not the value of the make variable,
when you run this:

    all:
            @echo M1=$$MAKEFLAGS
            @echo M2=$(MAKEFLAGS)

you get different output: the environment variable DOES contain the
extra option, but the make variable does not.  The behavior of these
things is subtle enough already, it seems unpleasant to make it even
more complex.

Then I figured, why not do the check during the make variable
expansion, regardless of whether it was for the child environment or
for another reason.  But even this is not great since you might be
saving that value for later, etc.  Also there's the entire situation
around MFLAGS.  So I didn't actually try to implement this.


So then I thought maybe it's time to junk the current jobserver
implementation of a simple pipe and go with something more
sophisticated that doesn't require being disabled when we invoke a
subcommand: something that only the make subcommand would care about. 
This would be nice since the jobserver could be inherited through
intermediate "non-make" commands.

So the first thing I tried was using POSIX semaphores, since the
Windows implementation uses Windows semaphores.  They're pretty
portable and easy to use.  In fact, I completely implemented it before
I ran into a problem: as you may recall the tricky part of the current
jobserver implementation is that we need to be able to wait for EITHER
a child job to exit (SIGCHLD) OR a new job token to appear (currently
read(2) on a file descriptor).  This required a lot of machinations.

Well, I don't see any possible way to do that kind of thing with
semaphores.  The are implemented as sem_t* and there's no facility I
can find that would provide a FD that could be used with pselect() or
whatever.  So there's no reliable way to wait for either one of these
events that ensures we don't miss a signal.  Basically we'd be reduced
to polling, which I _really_ don't want to do.


So, I guess the next thing to look at is named pipes rather than
anonymous pipes.  It's a little annoying because I have to manage the
named pipe file which means finding a temporary directory I can create
it in, dealing with permissions issues, and cleaning it up when I'm
done (of course this can't be cleaned if we are killed via SIGKILL or
SEGV or whatever).  However, it's good in that it gives the same
interface as the current pipes without worrying about the CLOEXEC /
trying to read from a descriptor that might be closed, or not, etc.

So I guess I'll try to work on that this weekend.

Cheers!