[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parallelism a la make -j <n> / GNU parallel
From: |
John Kearney |
Subject: |
Re: Parallelism a la make -j <n> / GNU parallel |
Date: |
Fri, 04 May 2012 21:02:27 +0200 |
User-agent: |
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 |
Am 04.05.2012 20:53, schrieb Mike Frysinger:
> On Friday 04 May 2012 13:46:32 Andreas Schwab wrote:
>> Mike Frysinger <vapier@gentoo.org> writes:
>>> i wish there was a way to use `wait` that didn't block until all the pids
>>> returned. maybe a dedicated option, or a shopt to enable this, or a new
>>> command.
>>>
>>> for example, if i launched 10 jobs in the background, i usually want to
>>> wait for the first one to exit so i can queue up another one, not wait
>>> for all of them.
>> If you set -m you can trap on SIGCHLD while waiting.
> awesome, that's a good mitigation
>
> #!/bin/bash
> set -m
> cnt=0
> trap ': $(( --cnt ))' SIGCHLD
> for n in {0..20} ; do
> (
> d=$(( RANDOM % 10 ))
> echo $n sleeping $d
> sleep $d
> ) &
> : $(( ++cnt ))
> if [[ ${cnt} -ge 10 ]] ; then
> echo going to wait
> wait
> fi
> done
> trap - SIGCHLD
> wait
>
> it might be a little racy (wrt checking cnt >= 10 and then doing a wait), but
> this is good enough for some things. it does lose visibility into which pids
> are live vs reaped, and their exit status, but i more often don't care about
> that ...
> -mike
That won't work I don't think.
I think you meant something more like this?
set -m
cnt=0
trap ': $(( --cnt ))' SIGCHLD
set -- {0..20}
while [ $# -gt 0 ]; do
if [[ ${cnt} -lt 10 ]] ; then
(
d=$(( RANDOM % 10 ))
echo $n sleeping $d
sleep $d
) &
: $(( ++cnt ))
shift
fi
echo going to wait
sleep 1
done
which is basically what I did in my earlier example except I used USR2
instead of SIGCHLD and put it in a function to make it easier to use.
Re: Parallelism a la make -j <n> / GNU parallel, Chet Ramey, 2012/05/04
- Re: Parallelism a la make -j <n> / GNU parallel, Mike Frysinger, 2012/05/04
- Re: Parallelism a la make -j <n> / GNU parallel, Andreas Schwab, 2012/05/04
- Re: Parallelism a la make -j <n> / GNU parallel, Mike Frysinger, 2012/05/04
- Re: Parallelism a la make -j <n> / GNU parallel,
John Kearney <=
- Re: Parallelism a la make -j <n> / GNU parallel, Greg Wooledge, 2012/05/04
- Re: Parallelism a la make -j <n> / GNU parallel, John Kearney, 2012/05/04
Re: Parallelism a la make -j <n> / GNU parallel, Mike Frysinger, 2012/05/04
Re: Parallelism a la make -j <n> / GNU parallel, John Kearney, 2012/05/04
Re: Parallelism a la make -j <n> / GNU parallel, Mike Frysinger, 2012/05/05
Re: Parallelism a la make -j <n> / GNU parallel, Andreas Schwab, 2012/05/05
Re: Parallelism a la make -j <n> / GNU parallel, Chet Ramey, 2012/05/07
Re: Parallelism a la make -j <n> / GNU parallel, John Kearney, 2012/05/05
Re: Parallelism a la make -j <n> / GNU parallel, Mike Frysinger, 2012/05/06
Re: Parallelism a la make -j <n> / GNU parallel, John Kearney, 2012/05/06