Re: Patch to allow make to load plugins that add new functions.

[Tim Murphy]

Hi,

On 5 April 2012 23:12, Paul Smith <address@hidden> wrote:
> Hi Tim;
>
> Before going too much further note that I've got a semi-implemented
> "load" operator in my source already, which fulfills a similar function
> except in a less sophisticated way: it just calls a function in the
> loaded object after loading and that function can do whatever it wants.
>
> I can send along a patch if you're interested in checking it out.  I
> have some regression tests added, documentation, autoconf, etc.
>
> Boris and I discussed this on the alpha list a number of months ago and
> his position was that we should not try to restrict things.  His
> suggestion was to follow the GCC model and just let people do what they
> wanted to.  The downside of this is that we wouldn't be defining a big
> stable API: we would reserve the right to change things internally with
> every release, which would potentially break loadable objects.

I see the value in a plugin system as being that I don't have to
recompile the plugins for every version of make. In a way it's tending
towards "why bother" if you did have to do that.  The idea that you're
describing probably wouldn't cause disruptions to most plugins but
there's no way of knowing except to test all plugins with all versions
of make anyhow to find out.  It would be a maintenance burden that's
quite similar to maintaining a custom make.

You also have to think about where in makes startup these things
happen. Some stuff needs to be done early, some later - if your plugin
loader needs a fully initialised make process (because it has to read
a makefile to do load) then although you might think it is infinitely
flexible, it does come with limits.

I have to admit to a secondary agenda in that I'm thinking about tools
like Electric Accelerator which mimic gnu make - I'd like them to be
able to implement plugins too so that my build system can be
accelerated ($$$$$) or free :-) depending on the users choice.  This
is why I was thinking of it being quite defined and with restricted
access and using versions so that it's clear what needs to be
supported and what's offered.

> There are some outstanding issues: the first one is that I'd like the
> "load" operation to behave like the "include" operation, in that make
> would try to rebuild the target of the load and, if it were rebuilt,
> re-exec itself to reload it.  That's not implemented yet.

I think this is cool in a way but sort of overkill - I would imagine
building plugins as little as possible.  You see, if this is the first
time you've built the plugin then you can't have tested it - so why
are you running a build with an untested plugin where you don't know
if it "meets the contract"?  I know that in practice it will just be
getting built because building from source is a common thing even for
code that has been built and tested in exactly the same form 100x
before, but I see this as being a capability that is beside the point
for my use cases.  I will find the discussion if it's archived
somewhere and try to understand the case for it - my perspective might
be quite limited.

> Second, the way in which the objects are found for loading.  Do we just
> use the default dlopen() process?  Or do something more (or less!)
> sophisticated?

I don't know but dlopen and windows LoadLibrary are similar enough
that the same general methodology will work.  I'm not up enough on
alternatives to know if there's anything better.

> Finally, as you've noted in order for this to be useful generally we
> need to start publishing some header files.  What goes into them and how
> they get published is something of an open question.  It's possible that
> this will require a significant reworking of the current make header
> files (which would be OK with me since they're pretty messy currently).
> We could choose the public interface through what structures and
> functions we put into the publicly-facing header.

I have thought about this and it's harder if everything's to become
available for use and easier if one is defining some subset of
exported stuff.

A totally different integration mechanism might involve being able to
embed GNU make into other programs and talk to it and I see that as a
very interesting possibility which would become much easier with the
same kind of header reorg that a plugin mechanism with full access
might need.

> The most obvious interface is "define a new function", but others could
> also be useful: eval, set variables, etc.

Yup.  I wondered about how make macros are (ha ha mostly) functional
so I was thinking about starting with "reading variables, targets,
etc" because it's completely uncontroversial and was going to leave
"setting" or any other side effect for a while longer since it's the
kind of area where I would be most likely to make a mess.

Regards,

Tim


-- 
You could help some brave and decent people to have access to
uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/