Re: [Fab-user] Code review for changes in 1.x

[Michael Gliwinski]

On Monday 14 Feb 2011 17:05:27 Travis Swicegood wrote:
> address@hidden> wrote:
> > Hmm, I see your point, however I'm wondering if it might be better then
> > to formalize and clean up that registry first...
> 
> We could -- that was against the idea behind this style.  I wanted to get
> something functioning first, then we're refactor under the hood.  A @task
> is a @task is a @task as far as the fabfile user/writer is concerned.  How
> it's stored internally is something beyond their concern. Now, the
> namespaced tasks are a different story.

Yeah, what I meant below is that indeed it's good as it is for now as it 
preserves BC, etc.  Good point about it being hidden so an under the hood 
refactor should be easy later.

> > Fair enough, after a second look it seems to me a lot of that
> > loading/scanning
> > code would stay mostly as is to preserve BC even if we did have a nice
> > formal
> > "registry of tasks".
> 
> We could.  I thought about going this route and only scanning if tasks had
> been registered.  But what happens when you import a third-party module
> that uses @task and you don't?  Now there's things in the registry, so we
> can't know for sure if you've registered all of your tasks or not.

OK, I see your point.

> > > > I like the namespacing implementation, although because "explicit is
> > > > better
> > > > than implicit" I agree with Jeff's comment to #56 [2] about having a
> > > > register_subtasks or similar function.
> > > 
> > > Let me think on this one.  My initial inclination leans slightly toward
> > > putting it on the task designer to specify what can be used as a task
> > > and what can't, but I can definitely see the point of it feeling a bit
> > > magic.
> > 
> > I agree!
> > 
> > Hmm, I'll think about it some more too.  I think the designer should
> > specify
> > where the tasks are but something doesn't feel right about having to
> > explicitly trigger scanning a submodule.
> 
> I would be open to starting the registry refactor and using a
> fabric.api.register_as_task_module() that would work the same as the
> variable does.  Inside the vendor.tasks.apache, for example, instead of
> FABRIC_TASK_MODULE, they would import register_as_task_module() and call
> that.  Then the scanning code could check variables against a list of
> registered modules rather than a simple variable check.
> 
> I did contemplate going that route, but landed on the variable simply
> because it was the simplest solution.

I was thinking of namespaces as means of splitting fabfiles in this case.  See 
below for my general reasoning on shared functionality.

> >  >  My reasoning for putting it in the module itself, rather than the
> > 
> > fabfile
> > 
> > > is that the module gets written once, the fabfiles get written multiple
> > > times.  It seems reasonable to write it once, rather than having to
> > > explicitly register it each time.
> > 
> > Ah, OK, just noticed we are probably thinking about different use cases. 
> > I see namespaces as just a neat way to partition a fabfile into
> > components (I have some huge fabfiles which could use that :))  I.e.
> > parts of one fabfile package related to a particular
> > system/project/domain.
> > 
> > For shared functionality between different fabfiles I just use a separate
> > Python package (here's where setuptools-like extending of fabric
> > namespace would be useful though).
> 
> There are two use cases.  The one you mentioned (fab production.deploy, fab
> staging.deploy).  I use that quite a bit.
> 
> The other case that I'm hoping this encourages is shared stuff.  For
> example fab apache.restart, fab django.init, and so on.  This would make
> it possible for people to package common tasks and distribute them that
> way.  I know of several developers (myself included) who started down this
> path only to discover we had to use PEAR-style prefixing for everything.

Wouldn't shared functionality rather be something that you use in tasks in 
your fabfile rather than exposing it as tasks on command line?  I mean, yeah 
there are few small and simple bits like <some_service>.restart or shell or 
even wrappers for some remote commands you may run interactively like grep, 
etc.  but most of shared stuff I have is utilities/objects I only use in 
actual fabfile tasks.

For example: I have a shared RPMBuilder class which wraps mock on a build 
server to build RPMs, I do not expose it directly on fab command line but 
instead in each project's fabfile I have a command/task which calls this 
RPMBuilder with options specific for this project.

In those cases where a shared function makes sense to be used directly as fab 
command I think I'd rather want to be able to control the fact that it's 
exposed from the particular fabfile being used.

Do you have lot of shared functionality which is exposed directly?

> > > One point, the register_subtasks code suffers from the same problem
> > > that lead to scanning for @task's rather than having @task actually
> > > register
> > 
> > --
> > 
> > > we currently scan the file for tasks rather than having an explicit
> > > registry to stuff things into.  Now, it could be that we import the
> > 
> > Hmm, actually having to scan submodules one way or another is the
> > problem. Still we have to do that for BC.  Maybe just being able to
> > limit where to scan?  (aka __all__ for submodules to include)?
> 
> And this would exist in the fabfile?

Yeah, as per above, assuming fabfiles are specific to a project or domain, 
it's about being able to control what each particular fabfile exposes.

Look at it this way: modules/packages with shared functionality have to be 
installed globally (specifically somewhere on Python's path so it's accessible 
from different fabfiles), now for example you have a module 'apache' which 
provides ways to start, stop, configure, check status of the server, etc.; now 
you have a fabfile which is meant to be used by support group for diagnostics 
of various systems including an apache server; if you import that shared 
apache module to get the status command all of a sudden you also get one to 
reconfigure the server.


-- 
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

**********************************************************************************************
The information in this email is confidential and may be legally privileged.  
It is intended solely for the addressee and access to the email by anyone else 
is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or 
any action taken or omitted to be taken in reliance on it, is prohibited and 
may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail 
are subject to the terms and conditions expressed  in the governing client 
engagement leter or contract.
If you have received this email in error please notify address@hidden

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, 
BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************