Re: Parallelism a la make -j <n> / GNU parallel

[Mike Frysinger]

On Friday 04 May 2012 15:25:25 John Kearney wrote:
> Am 04.05.2012 21:13, schrieb Mike Frysinger:
> > On Friday 04 May 2012 15:02:27 John Kearney wrote:
> >> Am 04.05.2012 20:53, schrieb Mike Frysinger:
> >>> On Friday 04 May 2012 13:46:32 Andreas Schwab wrote:
> >>>> Mike Frysinger <vapier@gentoo.org> writes:
> >>>>> i wish there was a way to use `wait` that didn't block until all the
> >>>>> pids returned.  maybe a dedicated option, or a shopt to enable this,
> >>>>> or a new command.
> >>>>> 
> >>>>> for example, if i launched 10 jobs in the background, i usually want
> >>>>> to wait for the first one to exit so i can queue up another one, not
> >>>>> wait for all of them.
> >>>> 
> >>>> If you set -m you can trap on SIGCHLD while waiting.
> >>> 
> >>> awesome, that's a good mitigation
> >>> 
> >>> #!/bin/bash
> >>> set -m
> >>> cnt=0
> >>> trap ': $(( --cnt ))' SIGCHLD
> >>> for n in {0..20} ; do
> >>>   (
> >>>           d=$(( RANDOM % 10 ))
> >>>           echo $n sleeping $d
> >>>           sleep $d
> >>>   ) &
> >>>   : $(( ++cnt ))
> >>>   if [[ ${cnt} -ge 10 ]] ; then
> >>>           echo going to wait
> >>>           wait
> >>>   fi
> >>> done
> >>> trap - SIGCHLD
> >>> wait
> >>> 
> >>> it might be a little racy (wrt checking cnt >= 10 and then doing a
> >>> wait), but this is good enough for some things.  it does lose
> >>> visibility into which pids are live vs reaped, and their exit status,
> >>> but i more often don't care about that ...
> >> 
> >> That won't work I don't think.
> > 
> > seemed to work fine for me
> > 
> >> I think you meant something more like this?
> > 
> > no.  i want to sleep the parent indefinitely and fork a child asap (hence
> > the `wait`), not busy wait with a one second delay.  the `set -m` +
> > SIGCHLD interrupted the `wait` and allowed it to return.
> 
> The functionality of the code doesn't need SIGCHLD, it still waits till
> all the 10 processes are finished before starting the next lot.

not on my system it doesn't.  maybe a difference in bash versions.  as soon as 
one process quits, the `wait` is interrupted, a new one is forked, and the 
parent goes back to sleep until another child exits.  if i don't `set -m`, 
then i see what you describe -- the wait doesn't return until all 10 children 
exit.
-mike

signature.asc
Description: This is a digitally signed message part.