Re: [Gnu-arch-users] Re: Extension language

[Colin Walters]

On Fri, 2003-10-17 at 11:05, Tom Lord wrote:
> 
> You guys are weird.  We start with a casual mention of building an
> extension framework [...]

But, what exactly is an "extension framework"?  You originally said
"extension language", which I interpreted to mean "embed a programming
language inside libarch/tla", which I am completely against.

But the rest of your mail seems to be more just talking what kinds of
tools you could write using pretty much any programming language with a
nice binding for libarch.

So if you could clarify exactly what a "framework" is, that would be
helpful.

> Walters: good question (paraphrased: "What's the other `stuff' that
> would go in a framework besides just trivial command wrappers?").
> I think it's mostly an area for exploratory programming as in:

[...]

> But just speculatively, what might "stuff" be?  Some random ideas:
> 
> * namespace and log file parsers
> 
>   Yes, there's a _little_ bit of code in tla that already does
>   this but it's reasonable to implement that externally and
>   convenient to have output in whatever your favorite data structures 
>   are in the language you're working in.

Yes...this would be one of the first things you would do in implementing
a language binding.

> * a configurable mirror manager
> 
>   A tool that keeps some persistent state about what archives I like
>   to mirror and according to what rules.  It might let me throw a new
>   archive name/location at it and set up the mirror automatically.  It
>   might remember for which archives I like to limit the mirror and
>   which not, which I use --no-cached, etc.  It might remember
>   timestamps of when mirrors were last updated and permit (possibly
>   concurrent) runs as either a batch-mode that updates everything
>   needing updating or trigger-mode that updates a particular archive.

This seems more like an application you would write using a framework,
not something that would be included in the framework itself.

> * a tiny language for merge specs

Now this is interesting.

>   If someone says: "I have a change that implements FOO" th ey might be
>   talking about a change in a single patch, a change that needs
>   cherry-picking several different patches, or a change on some
>   version (or some revision) which relative to an indicated baseline.
> 
>   If I get news of such a change, and have a tree I want to apply
>   it to, the submitters description needs translated into actual
>   merge commands for my particular tree.
> 
>   So perhaps a little tiny language for specifying changes -- kind
>   of what pqm has already but further developed.

I think that here is a major confluence between a "smart server" and
pqm.  This hypothetical language could become the _protocol_ that is
used for communication, either over email to a pqm, or over a socket to
a smart server.  So there could be some code reuse here.

However, I'm not entirely sure that this protocol entirely fits in at
exactly the tla level.  A good example would be how using pqm you could
do "meta commits" to multiple archives at once.  That's something that
is easily implemented just by reusing the functionality that's already
in tla, there's no need to implement it inside tla itself.

>   A library that maintains a bunch of temporary trees, keeping track
>   of what each one is (is it a specific revision?  a revision with
>   local changes?  a revision with one or more tiny merge programs
>   applied? etc).   That let's me attach a name and notes to these and
>   can give a catalog of them.

This would be a good candidate for being implemented in libarch.  Or
perhaps some higher-level library, say like liboverarch.

>   Similarly for scratch changesets -- taking the basic idea of undo
>   and what-changed and letting me shuffle those changesets in and out
>   of a "clipboard" of scratch changes.

This seems more like an application than a framework component.

> * an extensible interface to testing
> 
>   With the tools above, I can easily build up scratch trees from high
>   level specs like:
> 
>      perform a test run naming it: "sync up with the four horsemen"

I think this could just be a shell script, really. 

> Hopefully one could build still higher-level tools on top of that.
> For example, let's suppose your merge-request email address gets a merge
> request.   The merge program is extracted from it along with any notes
> and stored in some database for me.   Meanwhile, the merge program is
> scanned to find out what archives it draws upon and that result is
> compared to the archives listed in my configurable mirror manager.
> That comparison might trigger some updates to selected mirrors and
> might even auto-register some new mirrors with suitable limits.
> Completion of updating the mirrors might then trigger setting up a
> scratch tree for that merge request and preparing a `what-changed'
> changeset for it --- all so that when you get up in the morning and
> stumble to the screen you can fire up a browser that says "Got a merge
> request from Rob.  It applies cleanly. [click to see the changes]
> [click to run tests on it] [click to commit] [click to prepare a tree
> for editting it]."   Oh, and if it had conflicts, it should send email
> back to the original submitter about those.

Ok, now I see what you are thinking.  That would be cool.  But it's not
an argument for embedding a language inside tla.  You could do all of
the above just as easily with a language binding to libarch, and then
use your favorite programming language on top.

Now, if we find later that multiple tools are reimplementing things like
mirror manager libraries, then that means it's a good candidate for
pulling down into a C library, probably distributed with tla too.

Different languages are good at different things.  For example, if you
wanted to implement the above using GTK+, you really want to use an
object-oriented programming style.  Doing anything nontrivial with a GUI
in a procedural manner is just painful.  If we artificially restrict the
tla extension libraries to say Scheme (which lacks OO features), then
we've also restricted the classes of applications that people can easily
build.  

OO is just one example.  Another one might be parsing email messages. 
Python has a great library for doing this (as it does for lots of things
actually).  So if I'm going to be doing a lot of email parsing, that
would make me (personally) strongly inclined to use Python.  But if we
were restricted to using "tlascript" or something, it is unlikely to
have as good an email parser.

In other words, we should aim for maximum flexibility, let various
languages and frameworks compete on their own merits, and then see what
happens.  If say a Perl binding and associated tools come to
predominate, then we "bless" that binding with the status of being
included along with tla (although not required).  As we learn more about
the problem space, we refactor some of the Perl extensions back down
into libarch/tla/liboverarch.

signature.asc
Description: This is a digitally signed message part