GUIfying the Guile REPL

[Neil Jerram]

Hi there.  Given the amount of recent interest in graphical Guile
REPLs (and related ideas), I thought that the list might be interested
in a brief analysis that I wrote about approaches to implementing
this.  The analysis is appended below.

One word of caution...  We are aiming, as Ariel has suggested, to
somehow combine the work of guile-repl and guile-gui.  However, since
guile-repl was already moving on while I was writing this, the text
below should not be taken as indicating how that integration will
proceed.

Comments are very welcome.

Regards,
        Neil


GUIfying the Guile REPL
-----------------------

1. Introduction

This short essay is an analysis of three approaches to providing
something like the Guile REPL in a "graphical user interface" (GUI)
application.  The approaches in question are:

- the (gtk event-repl) module in the guile-gtk distribution

- the guile-repl module in Gnome CVS

- my own guile-gui distribution.

I'll begin by describing neutrally how each approach works and what it
provides, then go on to discuss the architecture and ideas involved.

2. Descriptions

2.1. (gtk event-repl)

(gtk event-repl) implements a REPL that has some features in common
with the standard Guile command line REPL implemented in boot-9.scm.
Aside from basic REPL functionality, the most notable common feature
is the support for backtraces and debugging when errors occur, using
the primitives `start-stack' and `make-stack'.

The key difference between boot-9.scm and (gtk event-repl) is in how
the REPLs behave when there is no more input immediately ready.  In
boot-9.scm, the REPL blocks in the call to `read', and the REPL only
returns to its caller when the REPL is exited (i.e. completely
finished with).  In (gtk event-repl), on the other hand, the REPL
returns to its caller whenever there is no more input available, and
it is the caller's job to obtain more input and then invoke the REPL
engine again by calling `repl-input'.

(gtk event-repl) does not actually implement a GUI for this REPL
engine, but it is very clear how one would do so.  Namely, by creating
a GUI text entry element such as a GtkEntry widget, and arranging to
call `repl-input' with the entered text every time the user presses
Return.  Note that a standard command line based REPL can also be
implemented using the (gtk event-repl) engine by a loop of the form:

  (let loop ((line (read-line)))
    (repl-input repl line)
    (loop (read-line)))

2.2. guile-repl

guile-repl creates a new Gtk+ widget, called `GuileRepl', that
consists of two GtkText widgets in a GtkVBox.  One of the GtkText
widgets - the `editor' - is used to input Guile expressions for
evaluation.  The other GtkText widget displays the result of the
evaluation.  Expressions are evaluated by calling
`gh_eval_str_with_standard_handler'.

[Update] guile-repl now uses gh_eval_str_with_catch and a handler that
displays errors in a GtkDialog.

The `editor' widget implements parenthesis matching and command line
editing features in C code derived from Snd and SCWM.

[Update] guile-repl also now has syntax highlighting in the style of
DrScheme.

guile-repl also provides a Bonobo wrapper for the new `GuileRepl'
widget, so that it can be used as a Bonobo component by other
applications.

2.3. guile-gui

guile-gui creates a Gtk+ application window that consists (like
GuileRepl) of two Gtk+ widgets in a GtkVBox.  The lower widget is a
GtkEntry where Guile expressions can be entered for evaluation.  The
upper widget is a GtkText that displays the results of the evaluation.

The REPL in guile-gui is implemented by the `top-repl' defined in
boot-9.scm, i.e. the same code that runs when using Guile at the
command line.  The connections between this REPL code and the GUI
front end are forged by constructing Scheme input and output ports
around the GtkEntry and GtkText widgets respectively, and setting the
REPL's current input, output and error ports to be the newly
constructed ports.

guile-gui also adds a collection of Gtk+ signal handlers (written in
Scheme) to the GtkEntry widget that implement command line editing,
parenthesis matching and command history features.

3. Discussion

3.1. Architecture

The main architectural difference between these approaches is that in
(gtk event-repl) and guile-repl, the text entry is logically the
master entity, and calls some function to do evaluation (either
`repl-input' or `gh_eval_str_with_standard_handler') when it needs to,
whereas in guile-gui, the REPL code is logically the master, and the
text entry is used as an input port to provide input whenever the REPL
needs it.

The overall pattern of a REPL is to somehow obtain some input, then to
evaluate that input, and perhaps to print a result.  Independently of
any particular architecture, one could say that, however a piece of
input is obtained, the continuation of the processing for that input
is to evaluate it and maybe print a result.  Therefore, in the (gtk
event-repl) and guile-repl approaches, `repl-input' and
`gh_eval_str_with_standard_handler' are acting as continuations.

Compare this with guile-gui.  In order to provide input without
blocking the rest of the application, guile-gui's entry port captures
and stores the continuation of its `read' call, and runs a loop
including `gtk-main-iteration' until the user presses Return in the
entry widget; then it calls the stored continuation with the text
contents of the widget.

In a nutshell, then, guile-gui uses Scheme's continuation feature
explicitly, while (gtk event-repl) and guile-repl are architected to
use procedures that act effectively as implicit continuations.

One's judgment as to which architecture is better will therefore
depend on how comfortable one is with explicit Scheme continuations.
My view is that using continuations permits an implementation
architecture that reflects the logical relationships between
components --- the `slave' entry widget/input port provides input to
the `master' REPL --- whereas the implicit continuation approaches
require turning the architecture upside down so that things which are
naturally responses (specifically, the input obtained by calling
`read') must be seen as driving events.  If the `upside down'
architecture doesn't feel upside down to you, that's probably because
it's basically the same thing as `event-driven programming', and
because event-driven programming in languages without explicit
continuations has become so widespread over the last decade.  But try
to think back, and remember how weird `event-driven programming' felt
the first time that you had to do it!

3.2. Code duplication

In general, I believe that duplication of code and effort is a Bad
Thing (TM).  In the context of this essay, there are two kinds of
duplication to consider:

- "horizontal" code/effort duplication, between the different GUIfying
  approaches described here

- "vertical" code/effort duplication, between any of these approaches
  and other pieces of Guile infrastructure.

3.2.1. Horizontal duplication

Code-wise, there is very little horizontal duplication in these
projects.  If we look at REPL-implementing code, we see that

- (gtk event-repl) implements a complete new REPL engine

- guile-repl uses the function gh_eval_str_with_standard_handler

- guile-gui has no REPL code at all.

For GUI code,

- (gtk event-repl) has none

- guile-repl implements widgets, parenthesis matching and command line
  editing features in C

- guile-gui implements widgets, parenthesis matching and command line
  editing features in Scheme.

In my view there is sufficient difference between C and Scheme
implementations of similar features that it is _not_ a Bad Thing to
have both of them.  A particular point of interest is that the
parenthesis matching algorithms are completely different.

Whether or not effort has been duplicated depends on what one takes to
be the objective.  If one's objective is simply to provide some kind
of graphical REPL, then clearly effort has been unnecessarily
duplicated.  But if one's objectives are more precise, and are met by
one approach in particular, then the effort for that approach was
justified, and it is likely that someone else's objectives would
justify the other approaches as well.

3.2.2. Vertical duplication

Vertically, there is a clear duplication of code and effort in the way
that both (gtk event-repl) and guile-repl reimplement the Guile REPL
coded in boot-9.scm.

Once the decision is taken to roll one's own REPL code, the coding
implication is inevitable: the more one wants to emulate the features
of the original REPL, the more duplication is required.  guile-repl's
REPL implementation is a single GH function call, and so is hardly
significant in strict code duplication terms, but it follows that
guile-repl's REPL is correspondingly lacking in features when compared
to the boot-9.scm REPL.  (gtk event-repl) duplicates a lot more of the
code from boot-9.scm, and so gains some similar features, but there
are still significant points of difference.

The alternative is to find a way to leverage the boot-9.scm REPL as it
is.  This is what guile-gui does, and so guile-gui automatically
inherits all of that REPL's features.  A further benefit of
guile-gui's port-based design, and independence of explicit REPL code,
is that guile-gui continues to Just Work if an application is run from
the Guile REPL that implements its own internal command REPL.

3.3. Componentization

guile-repl goes further than the other approaches in defining a Bonobo
component interface for a Guile REPL and using the GuileRepl widget to
implement this interface.  This is cool!

If I understand correctly (analogizing from my experience with COM),
the specification of the Bonobo interface is distinct from its
particular implementation by the GuileRepl widget.  Therefore, given a
sufficiently tight interface specification, it should be equally
possible to implement this interface using components from the (gtk
event-repl) and guile-gui approaches.  I would be very interested in
this: ideally, the mechanism necessary to say "I want to provide an
implementation of this interface, and here it is" would be exported to
the Scheme level, so that no C code is required for a new interface
implementation.

On a detailed level, I think there is one issue with the specified
interface.  Namely, what is the relationship (current module etc.)
between Scheme expressions evaluated by the REPL inside the interface
and other evaluation performed by the application outside the
interface?

3.4. Combinations

To conclude, here are some ideas for combining various parts of these
approaches and for avoiding duplication in the future.

- Extend boot-9.scm's REPL code so that it can be used in a
  non-blocking way like the (gtk event-repl) REPL engine.  This should
  completely obsolete the (gtk event-repl) code.

- To avoid using explicit continuations (if so desired), use
  guile-gui's widgets with paren matching and history handlers (but
  not as a port) as the front end for a non-blocking REPL.

- Use (gtk event-repl) or the non-blocking extension of the boot-9.scm
  to provide a more sophisticated REPL than
  gh_eval_str_with_standard_handler for the guile-repl widget.

- Compare and maybe interchange parenthesis matching algorithms
  between guile-repl and guile-gui.

- Investigate implementing the GuileRepl Bonobo interface using
  components from guile-gui.