[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: bash leaks sh-np-NNN files and pipes in /tmp/ when command substitut

From: Yuri
Subject: Re: bash leaks sh-np-NNN files and pipes in /tmp/ when command substitution is used
Date: Wed, 11 Dec 2013 13:36:49 -0800
User-agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0

On 12/10/2013 23:29, Yuri wrote:
Some of my scripts use command substitution, and now I see that there are lots of files like these in /tmp:
prw-------   1 yuri   wheel          0 Dec 10 13:32 sh-np-1386738492
-rw-r--r--   1 yuri   wheel    3278909 Dec 10 14:54 sh-np-1386721176

Besides the obvious question why they aren't deleted in the end, two questions: Shouldn't bash delete the pipe files once the child process opened them? On most platforms the deleted pipe will function the same with the benefit that it will get deleted automatically. Why some of he files /tmp/sh-np-NNN are regular files, not pipes? When I look through code I see mkfifo call creates them. Shouldn't it always create fifos and not files?

Ok, I figured the details out.
The actual script that I run is this radio playing command:
stdbuf -i 0 -o 0 -e 0 mplayer http://kqed-ice.streamguys.org:80/kqedradio-ch-e1 | \
  stdbuf -i 0 -o 0 -e 0 tee >(grep "^XXX XXX" | prepend-time >> test.log)

prepend-time is a script:
gawk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; }'

The problem only occurs with stdbuf(1) commands inserted there, and not without them.
1: bash creates a FIFO /tmp/sh-np-NNNNN
2: bash unlinks this FIFO
3: tee is passed /tmp/sh-np-NNNNN as an argument, and it creates a regular file with the same name

This is the race condition.
Obviously, bash doesn't ensure that unlink is issued after the child command opons it, and not before that. Order of operations 2 and 3 should be reversed. And just waiting for a while isn't enough! bash should only unlink the fifo after it has been open by the child command.

Please also note, that this is on FreeBSD, which behavior might be different. Linux or other OSes might be getting lucky here with timing, not sure. FreeBSD also has its own version of stdbuf(1).
This bug needs to be fixed.

The original intention of stdbuf there was to make it all unbufferer. Whether this works or not isn't clear, but it doesn't matter for the purpose of this bug report.

Suggested solution:
On FreeBSD, for example, there is the kqueue(2) facility, which is able to wait on the event of "link-change": EVFILT_VNODE operation (takes file descriptor) allows sub-operation NOTE_LINK (The link count on the file changed)
So the sequence will be like this:
1. bash creates FIFO
2. bash opens FIFO
3. bash waits with kqueue(2) with EVFILT_VNODE/NOTE_LINK
4. bash launches the child command
5. child command opens the FIFO as stdio
6. bash catches the link-change event
7. bash unlinks FIFO

Linux and other OSes must have an equivalent facility.
Sounds like a fun project to do. -)
Absolute must-have if bash needs to support command substitution.


CCing to the maintainer of bash on FreeBSD

reply via email to

[Prev in Thread] Current Thread [Next in Thread]