Re: [bug #60774] make hangs on fcntl lock when using -O and stdout is /dev/null

[Paul Smith]

On Sun, 2021-07-25 at 13:11 -0400, Dmitry Goncharov wrote:
> On Sun, Jul 25, 2021 at 10:03 AM Paul Smith <psmith@gnu.org> wrote:
> > Writing to things like /dev/lp0 SHOULD be locked, IMO: you don't
> > want multiple concurrent writers to your printers or your terminal!
> 
> The user intentionally redirects make to a printer along with another
> tool. Should make be smarter than the user and refuse?

I suppose you mean in the same way Mike reported, the "other tool"
locks the device (e.g. printer) and now if you use -O then make's lock
prevents make from writing to it (or vice versa).

I have no problem with make refusing to write to a device that's locked
in general.  If users don't want that to happen, they can just not use
the -O option.  Or if they do want to use the -O option and they really
do want to write to the device even if it's locked, it's trivial enough
to hide it from make by introducing a intermediate step, like this:

  $ make | cat > /dev/lp0

It's clear that /dev/null is a special case, however, because the order
in which data is written to it cannot be detected so what's the point
in preserving any order?

> > To me the most compelling reason to change the current locking
> > behavior is not this: I agree with David that simply special-casing
> > /dev/null could be a good solution; if you're redirecting to
> > /dev/null why even HAVE a lock?
> 
> Mike wanted to specify -O and redirect to /dev/null. Again, why
> should the tool be smarter the user?

What I'm saying is that IF make can detect that its stdout is going to
/dev/null then it shouldn't lock at all, because it's not necessary to
do so in order to meet the goals of the -O option, and doing so
introduces unnecessary latency in the build.

The problem is, I don't know of any portable way of determining whether
stdout is going to /dev/null, or not.  The only way I know of to do it
is to check /proc/<pid>/fd/1 and that's pretty much just available in
Linux (and requires you to have procfs mounted which not all Linux
systems do).

> > The most compelling reason to change the current behavior is that
> > unfortunately BSD-based systems don't support locking file
> > descriptors that aren't real files: so if you pipe make's output to
> > something on a BSD-based system like MacOS or FreeBSD, for example,
> > you get a ton of error messages.
> 
> One mechanism to avoid these error messages would be to stat stdout
> and check S_IFREG.

Certainly I didn't mean to suggest that detecting this situation is
hard (actually the simplest way is to just check the error code when we
try to take the lock and "do something else" if it fails).  The
question is what to do about supporting -O on these systems.

> >      submake: ; $(MAKE) -C $@ >sub.out
> > The way the implementation is today, the outer make and the submake
> > magically have different "output lock domains",
> 
> This is an interesting situation.
> 
> i considered this behavior to be a violation of the contract. The
> make manual says that when -O is specified the output of the parent
> and children is synchronized.

I don't agree; the -O option tells make that the user wants to see the
output of each rule not mixed with the output of other rules.  That's
all it means: any other deeper or more technical meaning is an accident
of the current implementation and can't be relied on.

Here the output to stdout is synchronized, and the output to the
redirected file is synchronized.

There's no reason that those two disjoint sets of output need to be
synchronized WITH EACH OTHER to meet the goal of the -O option.  Given
the choice between allowing output to go to these two different
locations in parallel vs. serializing them with each other, I don't
think anyone would prefer the latter.

> make is a portable tool and this magic does not happen on windows.

That's true but there are, unfortunately, some edge cases where Windows
and POSIX systems behave differently.  As long as the builds succeed in
largely similar ways and the edge cases don't cause incorrect builds,
we live with them.

My goal is not to provide an implementation of make which supports only
the least common denominator of all platforms.  The goal is to provide
the best implementation of make for the GNU system, and provide best-
effort support for other platforms that people care about as well.