process substitution and wait()

[Daniel Kahn Gillmor]

Over in the "here strings and tmpfiles" thread, a distraction came up,
which i'm splitting out into a separate thread.  Please don't conflate
the two, i'm just looking for further clarity about process
substitutions and the wait builtin.

dkg and chet wrote:

>>>> https://bugs.debian.org/920455
>>>
>>> It wouldn't really affect that. The reason `wait' waits for process
>>> substitution processes is that they set $!, making them "known to the
>>> shell" and subject to wait without arguments.
>>>
>>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/wait.html#tag_20_153
>> 
>> This behavior is actually different between bash 4.4.18 and 5.0, but i
>> think this is a separate discussion, so i'll defer it to a different
>> thread to avoid confusion here :) 
> 
> Yes. It was reported as a bug against bash-4.3, back in 2015, and I agreed
> with it, so the behavior changed.
> 
> http://lists.gnu.org/archive/html/bug-bash/2015-03/msg00076.html

I understand why you made the change, i think.  I'm just observing that
the change that was made has an observable impact on any child process
that happens to be bash.

here's bash 5.0 taking 3 seconds to run a command that posh and zsh and
dash all run in 0 seconds:

$ for x in posh dash zsh bash; do printf 'shell: %s\n' "$x"; time -p dash -c 
"exec $x -c wait" 2> >(sleep 3); done
shell: posh
real 0.03
user 0.00
sys 0.00
shell: dash
real 0.00
user 0.00
sys 0.00
shell: zsh
real 0.00
user 0.00
sys 0.00
shell: bash
real 3.00
user 0.00
sys 0.00
$

Note that the subshell does no process subsitution whatsoever -- the
process substitution is all happening in the outer loop.

The change between 4.4 and 5.0 is that bash 5.0's wait builtin is using
waitpid(-1) here, but 4.4 is not bothering to call wait() at all
(presumably because it has no outstanding jobs).  It's not clear to me
how $! gets passed through to the dash-exec'ed bash subshell here
either; the environment doesn't appear to change.  Can you help me
understand what's happening here?


I note that this hanging behavior predates 4.4 for any non-bash program
that call waitpid(-1), because the substituted processes are the
children of the subprocess anyway:

For example, consider the following (dumb, deliberately trivial) program:

-----------------------
/* waiter.c - compile with: gcc -o waiter waiter.c */
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>

int main(int argc, const char *argv[]) {
  int wstatus = 0;
  pid_t pid;
  const char * arg1 = (argc > 1) ? argv[1] : argv[0];

  pid = wait (&wstatus);
  printf ("%s: pid: %d wstatus: %d\n", arg1, pid, wstatus);

  return 0;
}
/* end waiter.c */
-----------------------

Because the program does not fork, any normal run of this program will
emit a single line and terminate:

$ ./waiter
./waiter: pid: -1 wstatus: 0
$

And, if i invoke it with some simple process substitution, there's still
no problem:

$ ./waiter >(cat > stderr)
/dev/fd/63: pid: -1 wstatus: 0
$

But, as soon as i hook that process substitution up to a file
descriptor, it hangs indefinitely and produces no output:

$ ./waiter 2> >( cat > stderr)
[…hangs…]


That's pretty weird, but it behaves the same in bash 4.4 as well as 5.0
at least.

Why are the process substitutions children of the spawned process and
not children of the parent bash shell itself? As someone who writes bash
scripts that use the wait builtin, it's a little distressing to me that
their behavior changes based on whether they are exec()ed in a process
that already has children or not.

Thanks for helping me understand this better!  

         --dkg

signature.asc
Description: PGP signature