Re: [Gnu-arch-users] Re: Extension language

[Tom Lord]

You guys are weird.  We start with a casual mention of building an
extension framework to make high-level process infrastructure easier
to write and the next thing I know you're talking about mucking around
in libarch, changing tla from a short-lived to long-lived process
model, freezing apis ....  hey, while we're at it, we'd better make it
thread-safe, start maintaining a swig interface file (or, wait, will
Swig do CORBA and XMLrpc?), and pepper it with callbacks.

Miles: about parsing reltables, perhaps the awk features in SCSH would
help?  http://www.scsh.net/docu/html/man-Z-H-9.html#node_chap_8
(and I'm sure that other languages are all over that stuff, too.)
As for not getting conveniently formatted output from this or that
command:  that is the kind of change I'm perfectly happy to see go
into tla.

Stephen: You're absolutely right that interface instability raises a
barrier.  I do think we're pretty close to done with the destabilizing
CLI changes, though, and in the meantime, it's a subjective judgement,
but I don't things are _that_ unstable that major headaches will
result.  (Tez is keeping up, for example.)

Walters: good question (paraphrased: "What's the other `stuff' that
would go in a framework besides just trivial command wrappers?").
I think it's mostly an area for exploratory programming as in:

        1) decide to build high-level tool FOO and get a fairly
           clear idea of what it should do

        2) decide that part of the output of the work should be
           the beginnings of a framework

        3) write FOO

        4) pick a new value for FOO and goto (1)

and hope that the series really does converge.

But just speculatively, what might "stuff" be?  Some random ideas:

* namespace and log file parsers

  Yes, there's a _little_ bit of code in tla that already does
  this but it's reasonable to implement that externally and
  convenient to have output in whatever your favorite data structures 
  are in the language you're working in.


* a configurable mirror manager

  A tool that keeps some persistent state about what archives I like
  to mirror and according to what rules.  It might let me throw a new
  archive name/location at it and set up the mirror automatically.  It
  might remember for which archives I like to limit the mirror and
  which not, which I use --no-cached, etc.  It might remember
  timestamps of when mirrors were last updated and permit (possibly
  concurrent) runs as either a batch-mode that updates everything
  needing updating or trigger-mode that updates a particular archive.


* a tiny language for merge specs

  If someone says: "I have a change that implements FOO" they might be
  talking about a change in a single patch, a change that needs
  cherry-picking several different patches, or a change on some
  version (or some revision) which relative to an indicated baseline.

  If I get news of such a change, and have a tree I want to apply
  it to, the submitters description needs translated into actual
  merge commands for my particular tree.

  So perhaps a little tiny language for specifying changes -- kind
  of what pqm has already but further developed.

  It'd be nice to have commands suitable for use in an interactive
  editor for the (hopefully trivial) parsed form of such programs, the
  idea being that a GUI can give me menus and patch lists to "pick"
  from, and the merge spec is generated automatically from that.


* a scratch tree and scratch patch manager

  A library that maintains a bunch of temporary trees, keeping track
  of what each one is (is it a specific revision?  a revision with
  local changes?  a revision with one or more tiny merge programs
  applied? etc).   That let's me attach a name and notes to these and
  can give a catalog of them.

  Similarly for scratch changesets -- taking the basic idea of undo
  and what-changed and letting me shuffle those changesets in and out
  of a "clipboard" of scratch changes.


* an extensible interface to testing

  With the tools above, I can easily build up scratch trees from high
  level specs like:

     perform a test run naming it: "sync up with the four horsemen"

        start with the latest tla--devo--1.1
        apply the merge-request "miles' latest command renames"
        apply the merge-request "walters' latest test suite additions"
        optionally apply the merge-request "rob's new replay and commit flags"
        apply the merge-request "mt's latest bug fixes"
        run the "basic build and test" tests

   and come back later to get a result like:

        "rob's new replay and commit addtions" had merge conflicts
        and was optional (skipped)

        overall test status was FAIL (results stored as scratch-tree 
        note)


* write changeset tools

  Write tools to compose changesets, notice conflicts between
  changesets, and variance adjust changesets.

etc.

Hopefully one could build still higher-level tools on top of that.
For example, let's suppose your merge-request email address gets a merge
request.   The merge program is extracted from it along with any notes
and stored in some database for me.   Meanwhile, the merge program is
scanned to find out what archives it draws upon and that result is
compared to the archives listed in my configurable mirror manager.
That comparison might trigger some updates to selected mirrors and
might even auto-register some new mirrors with suitable limits.
Completion of updating the mirrors might then trigger setting up a
scratch tree for that merge request and preparing a `what-changed'
changeset for it --- all so that when you get up in the morning and
stumble to the screen you can fire up a browser that says "Got a merge
request from Rob.  It applies cleanly. [click to see the changes]
[click to run tests on it] [click to commit] [click to prepare a tree
for editting it]."   Oh, and if it had conflicts, it should send email
back to the original submitter about those.


-t