Re: [Fab-user] Wait for parallel task to complete

[Morgan Goose]

First have you read the tutorial? This is the section that will get
you in on host list defining, and point you the the more detailed
information that I link to below it:
http://docs.fabfile.org/en/1.4.1/tutorial.html#defining-connections-beforehand
http://docs.fabfile.org/en/1.4.1/usage/execution.html#host-lists

As to what you want to do. Each task will loop over it's list of hosts
to run on. So top down it's TaskA over all HostListA's hosts. Then if
no errors move to the other task with the same or unique host list
(defined either globally or per task).

>From what you're wanting to do you will construct the fabfile like this:

local = ['a','b']
remote = ['c','d']

def test:pass
def update:pass

@task
@sequential
def deploy():
    execute(test, hosts=local+remote)
    execute(update, hosts=local+remote)


Then all hosts will have the test task run on them before moving to
looping over the two hosts lists and running the update. You then call
it simply with:

$ fab deploy

And you never run the test and update tasks themselves. Nor do you
define anything to be parallel. As parallel runs on the task would
test on both hosts in parallel, and then deploy to both hosts in
parallel. This was, running sequential and explicitly defined to do
so, will fail fast on the test hosts if one dies. Because the host
list's order is honored.

-goose

On Wed, Apr 25, 2012 at 1:13 PM, anatoly techtonik <address@hidden> wrote:
> On Sat, Apr 21, 2012 at 7:51 PM, Jeff Forcier <address@hidden> wrote:
>> On Fri, Apr 20, 2012 at 5:31 AM, anatoly techtonik <address@hidden> wrote:
>>
>>> Is it possible in fabric to wait until a subtask completes on all
>>> servers successfully before moving to next step?
>>
>> You need to use execute() to treat subroutines as if they were full
>> fledged tasks. execute() is the machinery that says "take this
>> function and run it once per host in this list of hosts." Right now
>> that machinery is just applying implicitly to your deploy() task and
>> you're probably using env.hosts to set your host list.
>>
>> Remove env.hosts and maybe make that host list a role in env.roledefs.
>> Then you can do this:
>>
>>    env.roledefs = {'myrole': ['a', 'b', 'c']}
>>
>>    def test(): ...
>>    def update(): ...
>>
>>    def deploy():
>>        execute(test, role='myrole')
>>        execute(update, role='myrole')
>>
>> That should have the effect you want: "fab deploy" => first test()
>> runs once per host, and when it's all done, update() will run once per
>> host. deploy() itself will end up running only one time total -- it's
>> just a "meta" task now.
>
> It took time to realize the that execute() iterates over the global
> list of hosts. I expected that the following two to be equivalent, but
> they were not:
>
>  1. fab -H local,remote test update
>  2. fab -H local,remote deploy
>
> I used the script without roles:
>
>  def test(): ...
>  def update(): ...
>  def deploy():
>      execute(test)
>      execute(update)
>
> 1st execution variant is fully synchronous (i.e. next task doesn't
> start until previous finishes) and gave the sequence:
>
>  local test
>  local update
>  remote test
>  remove update
>
> but the 2nd variant with subtasks was confusing (I indented to see
> what's going on):
>
>  local deploy
>    local test
>    remote test
>    local update
>    remote update
>  remote deploy
>    local test
>    remote test
>    local update
>    remote update
>
> I found fabric pretty counter-intuitive in this case. I tried to fix
> that without roles by explicitly passing current host:
>
>  def test(): ...
>  def update(): ...
>  def deploy():
>      execute(test, host=env.host_string)
>      execute(update, host=env.host_string)
>
> This gives:
>  local deploy
>    local test
>    local update
>  remote deploy
>    remote test
>    remove update
>
> Still not the desired behavior. The desired is:
>  deploy
>    local test
>    remote test
>    wait
>    local update
>    remote update
>
> I've tried using @parallel decorator for deploy task and it seemed to
> work fine at first.
>  local deploy
>  remote deploy
>  remote test
>  local test
>  local update
>  remote update
>
> But the test step didn't not synchronize - local update executed while
> remote test was still running. It looks like the roles is the only
> chance, but I'd like to avoid hardcoding server names at all costs. Is
> it possible?
>
> --
> anatoly t.
>
> _______________________________________________
> Fab-user mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/fab-user