Re: Markup module patch (Issue 2026)

[Ian Hulin]

On 18/12/11 15:46, David Kastrup wrote:
> Ian Hulin <address@hidden> writes:
> 
>> This is to allow an attempt at separation of function so we don't
>> have to have every single scheme-level procedure defined in the
>> body of lily.scm itself.  However the vast majority of these
>> scheme files just get loaded and evaluated and the scheme
>> procedures are added to the (lily) module.
>> 
>> The difference in behaviour here between the two Guile flavours
>> is Guile V1.8 is less fussy because it allows a level of lazy
>> binding, which means may get away with forward referencing a
>> macro until the component scheme file is subsequently loaded and
>> defines the macro. Guile V2 won't stand for this at all.
>> 
>> My current plan for 1686 is to retain this way of building all
>> the (lily) definitions, but pre-compile the component scm/*.scm
>> files and load the resulting scm/out/*.go files.  Ideally I want
>> to generate these during the build using Guile V2 'guild
>> compile', but I'm prototyping at the moment with extra hooks in
>> our custom load routine to compile on-the-fly in the ly:load
>> procedure in lily.scm.
> 
> Ok, I might be really bad at interpreting this, but I think the
> problems might arise from trying to obey several identities for
> which I see no compelling reason.
> 
> a) identity of files with compilation units.  You make it appear as
> if every .scm file needed to get compiled into a separate .go file
> when it would appear reasonable to just compile scm/lily.scm which
> includes all the rest.
> 
No can do backward-compatibly before Guile V2.  Guile V2 has an
(include) and (include-from-path) which may do what we need as they
seem to work a bit like C #include, but they are not documented yet by
Guile.

Compiling the component scm/*.scm files with Guile V2 isolates
scheme-level problems a lot easier if any of these are thrown up at
compilation time.

A thought about build dependencies - obviously lily.go will depend on
lily.scm and all the component scm/*.scm which lily.scm loads.  What
about internal dependencies between the loaded .scm files, such as has
been thrown up by the markup code?  I remember thinking about this was
what prompted the original decision to go for a module for the markup
subsystem.

> b) identity of compilation units with Scheme modules (in the
> namespace sense).
> 
> If Guile v2 does not allow a compilation unit to be spread over
> more than a single file, and/or requires macros from a separate
> file to be loaded into a separate namespace, this would have
> appalling usability consequences for Guile in general.
> 
OK, Guile has an internal module directory, derived from (foo bar)
name declared in its (define-module) statement.  If Guile doesn't find
it in the internal directory, it falls back to converting it to a
path-and-file specification relative to the root current working
directory, or relative to elements in the current setting of scheme
list %load-path. So (include-modules (foo bar)) will look for
foo/bar.scm relative to somewhere off the directories in %load-path.
Typcally %load-path is set to (<lilypond-datadir>
<lilypond-datadir/scm) in the main.cc code, so (use-modules (scm
editor)) will find scm/editor.scm. Implicit in this conventions is
that the source file containing (foo bar) => source file foo/bar.scm.

You *might* split definitions for module (foo bar) between foo/bar.scm
and foo/baz.scm but you will need either to duplicate the
(define-module) declarations in both source files, or ensure
foo/baz.scm does something like (set-current-module (resolve-module
'(foo bar))) *and* make sure the tail of foo/bar.scm does a
conditional statement to (load-from-path "baz.scm") (for Guile V1.8)
or (include-from-path "baz.scm") (for Guile V2).  Additionally you
will need ensure the build has an explicit make rule similar to:

$(outdir)foo/bar.go : $(indir)foo/bar.scm $(indir)foo/baz.scm
    guild compile $(indir)foo/bar.scm --output-file=$(outdir)foo/bar.go


Yes, you can have a one-to-many relation between scheme objects (.go)
and scheme sources (.scm) but it's a PITA, slightly more complex, and
a maintenance complication.  I think it's preferable to have a single,
longer scheme source file and if the declarations fall into logical
sections to comment the sections as such.

> So either I interpret something wrong into what you are saying, or
> you interpret something wrong into what the Guile developers are
> saying, or the Guile developers interpret something wrong into what
> usability should be saying.  Or a bit of each.
> 

*Shrug*.  It's very easy to get things confused with two different
projects (Guile and LilyPond) developing at different speeds and with
fundamental deep-level changes going on in each (like byte-compilation
in Guile and your spring-clean on the parser stuff, also I only get
chance to work on Lily in bursts, and I'm not able to swap between
development with the two Guile versions as easily as I'd like.

Cheers,

Ian