Re: guile-emacs for gsoc

[Ken Raeburn]

Hi, Brian.  It's great to hear that you're interested in this project!

On Apr 7, 2010, at 15:55, Brian Templeton wrote:
> * Guile now has a partial implementation of Emacs Lisp, and it mostly
>  works although a few features are missing

Some of them are pretty significant.  I think the buffer- and frame-local 
variables are pretty crucial -- though we probably don't want the basic Guile 
elisp support to have "current buffer" and "current frame" as concepts.  
Something a bit more abstract, where code in Emacs could nail it down to 
looking for a specially designated "buffer" object or whatever, would be 
cleaner.

There are also certain classes of variables, only created in C code, which can 
only hold integer values, or only hold boolean values.  Like buffer-local 
variables, these are implemented in Emacs by storing special object types known 
to the evaluator in the "value" slot; those objects include pointers to C 
variables of appropriate types, which are also used directly by Emacs C code.

Scheme code that messes around with these special Lisp variables should 
probably see the same restrictions on new values, and have the same impact on 
the behavior of the C code.

There's also the chance that by the time we get through this work, Emacs will 
have added optional lexical scoping, and some measure of thread support.  Both 
of these projects are in the works now (in fact, I think the lexical scoping 
work may be finished and just waiting to merge for Emacs 24), and if we want to 
potentially be able to swap in Guile's elisp support someday, it'll need to 
behave the same, except perhaps in corner cases that don't matter -- but the 
Emacs developers would be deciding which cases matter.

You don't need all of Emacs to be able to reproduce the sort of environment 
where you can exercise these language features.  For example, a "smob" type 
(Guile's application-defined object type) called "buffer" could have a C 
structure associated with it that holds a few Scheme variables, and a C 
function called from Scheme could print out or return the values stored in the 
structure associated with a buffer object denoted by a global variable 
"the-current-buffer", which could be changed by calling "set-buffer", etc.  
This buffer type doesn't have to actually hold any character data for you to be 
able to work on the low-level Lisp stuff.

> * Ken Raeburn has done proof-of-concept work to make Emacs use Guile's
>  object representation and GC for Emacs Lisp, which is now outdated but
>  shows that it's feasible (and perhaps it could be forward-ported?)

It's in the works.  Emacs switched version control systems recently, and I 
started a new job around the same time, so it's been taking me a while.  But 
I've got a few (not directly Guile related, but helpful for this project) 
patches about ready to submit upstream, and am working on rebasing the Guile 
changes onto a more recent version from the new upstream repository.  Probably 
I should target guile 1.8 with that, since that's what the old version was 
working with; once that's working, then update to 2.0.

Once it's brought forward to a current Emacs code base, there are lots of 
smaller sub-projects to tackle -- convert Lisp strings use Guile strings (and 
figure out what to do with text properties); convert Lisp vectors; convert 
hashes (and especially get weak hashes right!); etc.

Would my rebased version be ready for someone else to hack on soon?  *sigh*  I 
don't know... I hope so.

> * Emacs Lisp interaction is pervasive throughout much of the Emacs C
>  code, but I don't know how well-abstracted it is

Not very, in some places.  That's been a big part of my project.  Among my 
just-about-ready changes are macro-izing access to some fields of Lisp symbol 
objects, instead of using hard-coded C struct field names.

Another possible side project that's cropped up is making Emacs work with the 
Boehm GC library that Guile 2.0 will use.  There's an old development branch in 
Emacs to do that, but it never got folded in to the trunk.  I'm not sure if 
that's going to be needed to get Emacs to use Guile 2.0.

> * Once Guile's Emacs Lisp support is sufficiently good, Guile can be
>  swapped in for the Emacs Lisp evaluator
> 
> I'm not yet familiar with the relevant bits of Guile and Emacs internals
> (though I've worked on other Lisp implementations), and I only started
> following Guile development recently. So my question is, where could I
> best focus my efforts for two months of full-time work (plus a month of
> relatively free time before GSoC officially begins)? One obvious project
> would be to finish up the Emacs Lisp implementation; I just started
> looking at it last night, so I don't know how complete it really is.

> Another might be to go ahead with replacing the Emacs Lisp evaluator
> even if Guile's Emacs Lisp implementation is not quite complete, enough
> to get it booting (and then improving the Emacs Lisp language
> implementation would have immediate benefits).

Interesting idea... but I think we're not quite ready to meet in the middle yet.

My Guile-Emacs work has "smob" types for Lisp symbols and strings, because (1) 
they're somewhat different from Scheme ones, and (2) I haven't finished 
abstracting the C bits well enough.  So the Guile elisp code would work on 
Guile strings and symbols, and the other Emacs code needs smobs for now....

There's probably a lot of code comparing for equality with "nil" that would 
need tweaking to handle Guile's approach to the whole nil/()/#f issue.

As Ludo' indicated, just *how* we should meet in the middle is still an open 
question...

> And then there are side
> projects like writing a bytecode interpreter.

Well... Andy's right about compiling elisp source being the biggest win.  But 
from personal experience, I don't think it's good to assume that the .el file 
is always handy.  Although it's binary in form, overall I think it'd be a much 
simpler "language" to compile than Scheme or Lisp.  It doesn't have to be 
blindingly fast, but I think maybe we should consider supporting it...

Ken