Re: Patch to allow make to load plugins that add new functions.

[Tim Murphy]

Hi :-)

On 6 April 2012 01:16, Paul Smith <address@hidden> wrote:
> On Thu, 2012-04-05 at 23:59 +0100, Tim Murphy wrote:
>> I see the value in a plugin system as being that I don't have to
>> recompile the plugins for every version of make. In a way it's tending
>> towards "why bother" if you did have to do that.
>
> Well, this kind of combines with my other issue regarding automatic
> remaking of objects :-)
>
> The idea is that you wouldn't have a predefined set of make plugin
> OBJECTS that you distributed.  Instead you'd keep the SOURCE CODE for
> your plugins for your build environment right there in your source tree,
> and build them as part of your build process.  If they are automatically
> recompiled when out of date, this would be seamless.

I get it - a package can have it's own little collection of plugins
and keep them in source control with everything else.

> I have a hard time envisioning massive, complex plugins: most likely
> they'll be small, straightforward functions that perform targeted
> operations more efficiently than make (or Guile) can do them.  Such
> things would only take a couple of seconds to build and would be
> unnoticeable in any serious build environment.

I don't see any problem with performance at all.  I can see that with
some build systems that generate makefile it's not going to be easy to
have a little pre-build step where you build such plugins and thus
you're going to want to be able to trigger them from the main build
because that's all there is.  So this makes more sense to me now.
Since I don't personally need it, I'd be  thinking of that as the
feature to put in after an environment variable/commandline option
since this lets the feature get out there and get tested now by some
of us and it doesn't preclude the addition of new syntax.

>> You also have to think about where in makes startup these things
>> happen. Some stuff needs to be done early, some later - if your plugin
>> loader needs a fully initialised make process (because it has to read
>> a makefile to do load) then although you might think it is infinitely
>> flexible, it does come with limits.
>
> Heh, I didn't say "infinitely flexible"; that's too much to ask.  We
> have the --eval command line option already.  I'm open to adding an
> environment variable if someone can describe a powerful use-case.
>
>> I have to admit to a secondary agenda in that I'm thinking about tools
>> like Electric Accelerator which mimic gnu make - I'd like them to be
>> able to implement plugins too so that my build system can be
>> accelerated ($$$$$) or free :-) depending on the users choice.  This
>> is why I was thinking of it being quite defined and with restricted
>> access and using versions so that it's clear what needs to be
>> supported and what's offered.
>
> Hm.  I'd need to understand better what exactly would be involved here.
> What kind of plugins would be used for this and what would they do?  I'm
> not that jazzed about designing in a vacuum, especially just to enable
> proprietary software to work better with GNU make :-).
>
> What I really DON'T want to do is provide a locked down interface, which
> then turns out to be too restrictive to be useful and we're constantly
> fielding requests to open this or that, and having to worry about
> backward compatibility, etc.  I'd prefer to leave things open, at least
> for a while until we get some feedback/experience.  Yes, this does shift
> the burden away from me and onto the plugin writers (where it belongs of
> course!! ;-)), but it's not like there are so many versions of make
> floating around that this is that onerous.

Most programs which are not designed with the idea that they will be
used as libraries or that they will have lots of plugins are not
really structured to make it easy.  Make seems to be a mixture like
anything - the stuff about adding functions is very nice but, for
example, there is no clear way to use plugins to change the
"uptodateness" of files to use md5 as discussed in another thread. So
there is a "nice" portion and a not so nice portion. My major interest
is in having an easy life and since I've been a makefile writer most
of the time, I am looking for whatever makes that easier. If I have to
go to the extra trouble of writing plugins then I want them to not be
a constant source of change and I want them not to fail mysteriously.
Basically I want to use the "nice portion" of Make's API and know that
make developers have some degree of commitment to keeping it working
as is.  I don't want to dip into the "nasty" bit of make because that
can hardly ever be made to do what I want except by extreme trickery
which might work now but become unworkable in the next release.

As for other tools like Electric Make, I want to be able to arm-twist
them to support plugins (politely of course, Eric :-) ) without them
being able to say it's "impossible" or unreliable or that it requires
them to simulate the volatile internals of make to some infeasible
degree.  I realise that this is just my interest but it makes the
difference between bothering to use a plugin mechanism and not
bothering at all since it couldn't offer me a uniform makefile that I
could use across systems - I'd solve all my worries on GNU make and
still be busted on emake and since GNU make doesn't offer that amazing
filesystem which tells you when you have parallelsim problems and
corrects them. In other words I can't give up on emake without
converting 100s of developers into experts in writing good parallel
makefiles and inventing a way to get make to work on clusters of
machines (without pvmgmake's horrendously complicated setup).   People
are so much more difficult than software.

So my vote, FWIW, is to define what one does expect make to offer
plugins (the "nice" bits of make) and define what it definitely
doesn't and won't in the near future.  Anyone who wants a lot more is
probably going to have to customise make itself anyhow.  I think the
plugin should say what "environment" it expects and make should decide
if it can offer that.  If make fails on something it claims to support
then it's a bug on make otherwise it's clearly a plugin problem etc.

As you say with expectations, one has to set them at some point.

> I was thinking of marking this feature as "experimental" in the first
> release (in the documentation), just to be more clear on expectations.

Very very much so - there are many platforms to support anyhow and
when someone eventually tries plugins on them we might see new
problems so better to let it take time to mature.

Regards,

Tim


-- 
You could help some brave and decent people to have access to
uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/