Re: Some "fun" last weekend

[Dmitry Goncharov]

On Tue, Jul 19, 2022 at 6:13 PM Paul Smith <psmith@gnu.org> wrote:
> There are at least two Savannah bugs about this very subtle and
> annoying issue.

Do you mean that jobserver-auth is present in the child env, even
though the fds are closed?


> So the first thing I tried to do was to add another setting of the
> --jobserver-auth variable assignment to MAKEFLAGS of the child
> environment if we're not in the "submake context".  So it would look
> like this:
>
>   MAKEFLAGS= --jobserver-auth=2,3 --jobserver-auth=-2,-2
>
> where the "-2,-2" was there to override the original, and this would be
> added only when starting a process that was not a recursive make.
>
> This actually worked pretty well, but it's got some pretty frustrating
> quirks (I think I alluded to these in the Savannah bug).  Mainly,
> because we modify the environment not the value of the make variable,
> when you run this:
>
>     all:
>             @echo M1=$$MAKEFLAGS
>             @echo M2=$(MAKEFLAGS)
>
> you get different output: the environment variable DOES contain the
> extra option, but the make variable does not.  The behavior of these
> things is subtle enough already, it seems unpleasant to make it even
> more complex.

Looks good to me. At least i'd prefer a pipe and this (or similar)
quirk over all the disadvantages of named pipes.


> So the first thing I tried was using POSIX semaphores, since the
> Windows implementation uses Windows semaphores.  They're pretty
> portable and easy to use.  In fact, I completely implemented it before
> I ran into a problem: as you may recall the tricky part of the current
> jobserver implementation is that we need to be able to wait for EITHER
> a child job to exit (SIGCHLD) OR a new job token to appear (currently
> read(2) on a file descriptor).  This required a lot of machinations.
>
> Well, I don't see any possible way to do that kind of thing with
> semaphores.  The are implemented as sem_t* and there's no facility I
> can find that would provide a FD that could be used with pselect() or
> whatever.

If you are willing to have a platform specific impl, then linux has eventfd.
https://man7.org/linux/man-pages/man2/eventfd.2.html.

One portable impl to have a semaphore with a fd is to use a pipe.
sem_init opens a pipe.
sem_post writes one byte to the pipe.
sem_wait blocks on reading one byte from the pipe.

However, any of these still require that the mechanism survives fork and exec.
Named pipes with all their disadvantages survive fork and exec.

regards, Dmitry