New module system

[Mikael Djurfeldt]

Given the limited resources we have to conduct the research, design
and development that would be required to develop a good module system
of our own, and given the excellent quality of Dybvig's new module
system, I propose that we simply take Dybvig's system and adopt it for
Guile:

  http://www.cs.indiana.edu/~dyb/papers/popl99.ps.gz

We would probably make a few extensions, though.  One such extension
is that we'd like to handle hierarchical module names which can be
mapped onto a corresponding file hierarchy, the way we do now.

Probably, the nicest way to do this is to define a new set of syntax
handling Guile modules and write these macros in the language provided
by Dybvig's system.

While Dybvig's system has many arguments speaking for it as a
candidate module system for Guile (e.g. that it mixes very well with
the syntax-case macros), the most controversial point is that it might
turn out that the best way to implement it requires using information
from previously compiled modules when expanding a new module.

That is, when compiling (or interpreting for that matter) one module,
we need some macro definitions from the external modules we import
from.  The system would not use the source code of those modules, but
instead the object code produced by the compilation, or some other
kind of file produced when expanding a module.

We can look upon these files as corresponding to the pre-compiled
headers of some C development environments.

The main reason why we need this is the power and expressivness of
Dybvig's system: Since the macro expanders have the full power of the
Scheme language available to them, it can happen that some expanders
are non-deterministic---they might have different expansions at
different times of evaluation, which means that we can't just use the
expander definitions a second time to deduce how to connect bindings
across modules when importing.

Here are a few points that I'd like us to discuss:

* Do we want to use Dybvig's module system?

* How would we like to extend it for Guile?  Specifically, how should
  the Guile meta-modules (the new set of syntax for defining Guile
  modules) look like?

* Is it OK to generate expansion information in a separate file?
  If not, is there some way we can avoid it?  (We could for example
  restrict the language available to the expanders so that they behave
  deterministically.)