Re: New module system option :duplicates

[Mikael Djurfeldt]

Marius Vollmer <address@hidden> writes:

> The core model of such a system (but not its UI) is in the text
> workbook/compilation/new-model.text.  Mikael, could you try to
> consolidate your duplicate checking with the model laid out in that
> text?  I.e., can your :duplicate option be implemented on top of that
> model?

I've now read new-model.text and think it is very good.  I have some
smaller comments, though, which I'll give in a separate letter.

If I haven't missed something, there are no problems with implementing
the current :duplicate option within the framework of new-model.text.
In fact, while I had expected to need to use type information supplied
by the module system, your suggested framework allows the current
processing of the :duplicates option to continue to work in the same
way as currently.

The processing of the :duplicates option will be done in
process-define-module-for-compiler or process-define-module (as is
currently the case).  If I've understood your model correctly, this is
also where the partially constructed module processes its list of
imported modules.  When duplicate imported bindings are found, a set
of new bindings (an extra interface) is inserted at the front of the
use-list.  In the case of merging generic functions, these bindings
will have their own variables containing their own
<extended-generic>:s.  Thus, the compiler will never notice anything
unusual.  It will look just as if the generic function is imported
normally from a single module on the use-list.  (And this is, in fact,
exactly how merging is done in the current implementation.)

> Also (and I think new-model.text will have to change for this), we
> might want to have the concept of "potentially imported" bindings:
> when importing a set of bindings with unspecified names (such as "all
> of (bar)" as opposed to "the binding with name mumble of (bar)"),
> these bindings are only potentially visible.  Only when a binding is
> looked up in the current module will these bindings be considered and
> only then wehn none of the "actually imported" bindings matches.  When
> a potentially imported binding is found in this way, it becomes
> actually imported.  When more than one potentially imported binding
> matches, an error is reported (or the first one is selected in 'legacy
> mode').
>
> Such a model can easily support an interface like
>
>    (define-module (foo)
>      :use-module (blarg)
>      :use-module ((bar) :select (mumble))  ; explicitely import mumble
>      :use-module (baz)                     ; also imports mumble
>                                            ; but only implicitely and
>                                            ; since we have an explicit
>                                            ; mumble, we won't see it
>      :duplicates check)                    ; we expect no explicit duplicates

Do I understand this correctly in that you suggest having bindings
with two "priority levels": "explicitly imported" = priority 1,
"implicitly imported" = priority 2.  Only duplicate bindings with
priority 1 causes an error (:duplicates check) while other duplicate
pairs are resolved (at lookup-time) according to :duplicates first
with precedence for the binding of higher priority.

Hmm... Isn't this a little conceptually complex?  What is the gain of
differing between "potentially imported" and "actually imported"?
It seems like a nice thing to be able to determine the set of imported
bindings when the module is constructed.

Is the aim to support the current "lazy binding" technique used in
guile-tcltk?

> So in my view only ":duplicates check" should be added, and ideally be
> made the default.  How to orchestrate the transition is another
> issue...

My thought in adding "first" and "last" was to have a way of capturing
the current behavior of the module system explicitly, and to satisfy
the group of people who might like the "override" behavior.

Personally, I prefer "check", although the recently added "warn" might
give a more smooth behavior for users who like it.  E.g.

  :duplicates (warn last)

would behave as the current default, but, in addition, emit a warning.

Regarding the transition: What about making "check" the default
duplicates handler but provide a form which the user can put in
his/her .guile if he/she want to be backward compatible?

  (default-module-duplicates-handler 'last)

I've actually experimentally done and committed this.  (See further
below in the included NEWS entry.)

> Given that we have a way to get multiple conflicting symbols into the
> current module (via renaming), what about leaving generic merging to a
> separate facility like 'merge-prefix-generics'.  Actual code might
> look like:
>
>   (define-module (my-module)
>     :use-module ((math 2D-vectors) :prefixed 2d)
>     :use-module ((math 3D-vectors) :prefixed 3d)
>     :duplicates check)
>
>   ;; merge 2d-x and 3d-x into x.  Likewise for y and z.
>   (merge-prefix-generics (2d 3d) x y z)
>
> This is not much more tedious than ":duplicates merge-generics" but
> much more precise.  No unexpected mergings are done and the remaining
> collisions of non-generics are also dealt with in a useful way.

I like this.  However, I'd like to provide this facility *in addition*
to giving the license to merge.

Given that we already had `define-extended-generic' I've now added

  (define-extended-generics (x y z) #:prefix (2d: 3d:))

I chose this form because I have the feeling that we might want to
have other additional ways of selecting the generics in the future.

Please tell me if you like your `merge-prefix-generics' better or if
you have a third suggestion.

> Also, merging of generic functions might be a useful thing to do, even
> without using the module system explicitely.

Yes.  We've had `define-extended-generic' for a while, but I have not
come to documenting it in NEWS.  I'll fix my old sins ASAP.

> What about adding:
>
>   Eventually, we want to make ":duplicates check" the default.
>

Good idea.  As you can see below, the current default is

  (replace warn-override-core check)

> In summary, I think that ":duplicates check" is a very good thing, but
> I'm unconvinced about the rest of the handlers.  I'd rather see
> improvements to the selection/renaming mechanism.
>
> Does this make sense, Mikael?

Given my recent commits, I guess I have to return the question.  Do
you think my recent additions make sense?

>> The detailed rule for method visibility is this:
>
> I think it is wrong to talk about "method visibility".  The visibility
> of methods is not an important feature, one might even argue that
> methods don't have names at all.
>
> The thing that matters is the set of methods belonging to a certain
> generic function.  This set is unaffected by visibility issues.
>
> (I'm not saying that you don't understand methods and generic
> functions, Mikael, of course you do, more than anyone else here.  But
> still the use of terminology doesn't fit my mental model of generic
> functions and their methods.)

Agreed.  I'll change the terminology.

Best regards,
Mikael

Excerpt of new NEWS:

** Checking for duplicate bindings in module system

The module system now checks for duplicate imported bindings.

The behavior can be controlled by specifying one or more duplicates
handlers.  For example, to get back the old behavior (which was to use
the last imported binding of a certain name), write:

(define-module (foo)
  :use-module (bar)
  :use-module (baz)
  :duplicates last)

If you want the old behavior without changing your module headers, put
the line:

  (default-module-duplicates-handler 'last)

in your .guile init file.

The syntax for the :duplicates option is:

  :duplicates HANDLER-NAME | (HANDLER1-NAME HANDLER2-NAME ...)

Specifying multiple handlers is useful since some handlers (such as
replace) can defer conflict resolution to others.  Each handler is
tried until a binding is selected.

Currently available duplicates handlers are:

  check              report an error for bindings with a common name
  warn               issue a warning for bindings with a common name
  replace            replace bindings which have an imported replacement
  warn-override-core issue a warning for imports which override core bindings
  first              select the first encountered binding (override)
  last               select the last encountered binding (override)
  merge-generics     merge generic functions with a common name
                     into an <extended-generic>

The default duplicates handler is:

  (replace warn-override-core check)

** New define-module option: :replace

:replace works as :export, but, in addition, marks the binding as a
replacement.

A typical example is `format' in (ice-9 format) which is a replacement
for the core binding `format'.

** Merging generic functions

It is sometimes tempting to use GOOPS accessors with short names.
For example, it is tempting to use the name `x' for the x-coordinate
in vector packages.

Assume that we work with a graphical package which needs to use two
independent vector packages for 2D and 3D vectors respectively.  If
both packages export `x' we will encounter a name collision.

This can now be resolved automagically with the duplicates handler
`merge-generics' which gives the module system license to merge all
generic functions sharing a common name:

(define-module (math 2D-vectors)
  :use-module (oop goops)
  :export (x y ...))
                  
(define-module (math 3D-vectors)
  :use-module (oop goops)
  :export (x y z ...))

(define-module (my-module)
  :use-module (math 2D-vectors)
  :use-module (math 3D-vectors)
  :duplicates merge-generics)

x in (my-module) will now share methods with x in both imported
modules.

There will, in fact, now be three distinct generic functions named
`x': x in (2D-vectors), x in (3D-vectors), and x in (my-module).  The
last function will be an <extended-generic>, extending the previous
two functions.

Let's call the imported generic functions the "ancestor functions".  x
in (my-module) is, in turn, a "descendant function" of the imported
functions, extending its ancestors.

For any generic function G, the applicable methods are selected from
the union of the methods of the descendant functions, the methods of G
itself and the methods of the ancestor functions.

This, ancestor functions share methods with their descendants and vice
versa.  This implies that x in (math 2D-vectors) can will share the
methods of x in (my-module) and vice versa, while x in (math 2D-vectors)
doesn't share the methods of x in (math 3D-vectors), thus preserving
modularity.

Sharing is dynamic, so that adding new methods to a descendant implies
adding it to the ancestor.

If duplicates checking is desired in the above example, the following
form of the :duplicates option can be used instead:

  :duplicates (merge-generics check)