Re: Memory leak in wait

[Jean Delvare]

Hi Chet,

Thanks for the fast reply and the explanations, very appreciated.

On Thu, 06 Nov 2014 20:57:11 -0500, Chet Ramey wrote:
> On 11/6/14 8:09 AM, Jean Delvare wrote:
> > A memory leak has been reported in a bash script I maintain [1]. After
> > investigation, I was able to shrink the test case down to:
> > 
> > while true
> > do
> >     sleep 1 &
> >     wait $!
> > done
> 
> This isn't a memory leak, and the memory use is bounded.  The shell, as

OK, this is in line with valgrind's claim that all allocated memory was
still reachable. But how bounded is it? The shell script for which the
issue was originally reported starts with about 3 MB of memory but
reaches 32 MB over time (I don't know which version of bash did that
though.) That seems to be a lot of memory just to record process exit
statuses.

> per Posix, keeps the statues of the last child_max processes.  It gets
> child_max from the process's resource limit (ulimit -v, 709 (?) on my
> system).  The list is FIFO, so when the number of background statuses
> equals child_max, the oldest statuses are discarded.

"help ulimit" says:

      -v        the size of virtual memory

the scope of which seems to exceed just the number of child process
statuses to remember?

> Posix allows the shell to discard saved status values if $! isn't
> referenced before another async job is started, but bash doesn't try
> to do that.
> 
> > The above loop has an always-rising memory consumption (RSS value as
> > reported by ps -o rss.) This one OTOH:
> > 
> > while true
> > do
> >     sleep 1 &
> >     wait
> > done
> 
> When you wait for all background processes, the shell, as per Posix,
> discards all saved statuses.

OK, thanks for the information.

So I think in the case of a shell script daemon, I have two options:
1* Use "wait" without argument. As we are always waiting for a single
   process (sleep) to finish, I suppose this is equivalent to "wait
   $?", right?
2* Limit the value child_max. But using "ulimit -v" for that seems to
   be a rather violent and risky way to achieve this?

Also three questions remain, if you would be kind enough to answer them.

1* Why is the memory consumption steady when using bash 3.0.16? Was
   this version of bash not recording process exit statuses yet?
2* If bash remembers the process statuses, how does one access the
   information? I couldn't find anything related to that in the manual
   page, but it's huge so maybe I missed it.
3* If I never call "wait" and thus never free the recorded exit
   statuses, does it mean that the system can't recycle the process IDs
   in question until the shell script exits? That would be a major flaw
   in my script :-(

Thanks again,
-- 
Jean Delvare
SUSE L3 Support