Re: [Fab-user] Wait for parallel task to complete

[anatoly techtonik]

On Sat, Apr 21, 2012 at 7:51 PM, Jeff Forcier <address@hidden> wrote:
> On Fri, Apr 20, 2012 at 5:31 AM, anatoly techtonik <address@hidden> wrote:
>
>> Is it possible in fabric to wait until a subtask completes on all
>> servers successfully before moving to next step?
>
> You need to use execute() to treat subroutines as if they were full
> fledged tasks. execute() is the machinery that says "take this
> function and run it once per host in this list of hosts." Right now
> that machinery is just applying implicitly to your deploy() task and
> you're probably using env.hosts to set your host list.
>
> Remove env.hosts and maybe make that host list a role in env.roledefs.
> Then you can do this:
>
>    env.roledefs = {'myrole': ['a', 'b', 'c']}
>
>    def test(): ...
>    def update(): ...
>
>    def deploy():
>        execute(test, role='myrole')
>        execute(update, role='myrole')
>
> That should have the effect you want: "fab deploy" => first test()
> runs once per host, and when it's all done, update() will run once per
> host. deploy() itself will end up running only one time total -- it's
> just a "meta" task now.

It took time to realize the that execute() iterates over the global
list of hosts. I expected that the following two to be equivalent, but
they were not:

  1. fab -H local,remote test update
  2. fab -H local,remote deploy

I used the script without roles:

  def test(): ...
  def update(): ...
  def deploy():
     execute(test)
     execute(update)

1st execution variant is fully synchronous (i.e. next task doesn't
start until previous finishes) and gave the sequence:

  local test
  local update
  remote test
  remove update

but the 2nd variant with subtasks was confusing (I indented to see
what's going on):

  local deploy
    local test
    remote test
    local update
    remote update
  remote deploy
    local test
    remote test
    local update
    remote update

I found fabric pretty counter-intuitive in this case. I tried to fix
that without roles by explicitly passing current host:

  def test(): ...
  def update(): ...
  def deploy():
     execute(test, host=env.host_string)
     execute(update, host=env.host_string)

This gives:
  local deploy
    local test
    local update
  remote deploy
    remote test
    remove update

Still not the desired behavior. The desired is:
  deploy
    local test
    remote test
    wait
    local update
    remote update

I've tried using @parallel decorator for deploy task and it seemed to
work fine at first.
  local deploy
  remote deploy
  remote test
  local test
  local update
  remote update

But the test step didn't not synchronize - local update executed while
remote test was still running. It looks like the roles is the only
chance, but I'd like to avoid hardcoding server names at all costs. Is
it possible?

--
anatoly t.