Re: Looking For Advice On Updating Code From Guile 1.8.8

[Massimiliano Gubinelli]

Dear Andrew,

 in my experience in transitioning code from 1.8 to 2/3 (which have essentially 
the same phasing model as far as I understand) you have to keep in mind two 
things:

1) In Guile 1.8 macros are expanded only when needed (so for example could do 
nontrivial computations at runtime), while now all macros go away all at once 
and the compiled (and thus executed) code do not have any macro.

2) all that it is needed for macro must be available beforehand, in particular 
if you use regular procedures in macros either you define them locally inside 
the macro or you have to wrap them in (eval-when (expand compile load) .... ) 
forms. Note that this particular form forces evaluation your code at least 
twice, so be aware of side effects. E.g. try to put a (display ...) inside and 
see that it prints twice (either at expand time and compile time or at expand 
time and load). This would cure your problem when compiling the second file:

(eval-when (expand compile load)
(define (process-name name)
 (symbol-append name '-tail)))

(defmacro blah (name)
 (let ((new-name (process-name name)))
   `(quote ,new-name)))

but iif you use only the function inside the macro you could rewrite it as

(defmacro blah (name)
 (define (process-name name)
   (symbol-append name '-tail))
 (let ((new-name (process-name name)))
   `(quote ,new-name)))

or with a let and a lambda.

This two remarks solve many problems, but again you could run into issues if 
your program uses a lot of side-effects inside macros (e.g. to populate tables 
of symbols). These side effects will happen at compile time and the data could 
not be available anymore at runtime (because Guile load the object file instead 
of reexpanding the macros). So for example the program run ok the first time 
(because it is expanded/compiled/executed) but not the second time (because 
then the object file it is loaded from the cache and executed and no macro get 
ever expanded again). 

As far as I see there is no much difference between 2 and 3 so if you manage to 
run on 2 it should run on 3.

Hope this helps,
Best
Max


> On 10. Feb 2021, at 22:49, Andrew Burgess <andrew.burgess@embecosm.com> wrote:
> 
> Hello,
> 
> I'm in the process of updating a body of old code from guile 1.8.8 to
> either guile 2 or 3.  Ideally it would be great if the final code
> could run on both version 2 and version 3.
> 
> When I first started looking at the problem I was hitting hundreds of
> warnings like:
> 
>  ..... warning: possibly unbound variable `blah'
> 
> along with lots of the expected unknown procedures as you might
> expect.
> 
> So, I started working through the issues.  In a (probably stupid) move
> I started passing `--no-auto-compile' on the guile command line.
> After a few days of hacking I actually managed to get the code running
> again.
> 
> But then I remembered about the `--no-auto-compile' and figured I
> should probably remove that.  At which point I ran into a few
> problems.
> 
> Here's a small (contrived) example, which I think is representative of
> at least the first big problem I need to work around.  Imagine two
> files:
> 
> --- START: loader.scm  ---
> 
> (define (load-files)
>  (load "loadee.scm"))
> 
> (load-files)
> 
> (display (blah abc))
> (newline)
> (newline)
> 
> --- END: loader.scm  ---
> 
> --- START: loadee.scm ---
> 
> (define (process-name name)
>  (symbol-append name '-tail))
> 
> (defmacro blah (name)
>  (let ((new-name (process-name name)))
>    `(quote ,new-name)))
> 
> --- END: loadee.scm ---
> 
> This works fine when run as:
> 
>  guile --no-auto-compile -s loader.scm
> 
> But, when with the compiler I get:
> 
>  $ guile -s loader.scm
>  ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
>  ;;;       or pass the --no-auto-compile argument to disable.
>  ;;; compiling /tmp/loader.scm
>  ;;; /tmp/loader.scm:6:9: warning: possibly unbound variable `blah'
>  ;;; /tmp/loader.scm:6:9: warning: possibly unbound variable `abc'
>  ;;; compiled <snip>
>  ;;; compiling /tmp/loadee.scm
>  ;;; compiled <snip>
>  Backtrace:
>             5 (apply-smob/1 #<catch-closure 1a751c0>)
>  In ice-9/boot-9.scm:
>      705:2  4 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>  In ice-9/eval.scm:
>      619:8  3 (_ #(#(#<directory (guile-user) 1b44140>)))
>  In ice-9/boot-9.scm:
>     2312:4  2 (save-module-excursion _)
>    3832:12  1 (_)
>  In /tmp/loader.scm:
>        4:0  0 (_)
> 
>  /tmp/loader.scm:4:0: In procedure module-lookup: Unbound variable: abc
> 
> My understanding of what's happening here is that macros are expanded
> at compile time, while load is a run-time thing, which is happening
> after compilation.  Hence why 'blah' and 'abc' are considered possibly
> undefined.
> 
> Then in the compiled code the '(blah abc)' has not been macro
> expanded, and so the argument is first being evaluated, which leads to
> the error.
> 
> I read the manual on 'Local Inclusion'[1] and this seems to line up
> with my understanding of the problem.  It even makes special mention
> that using (include "...") instead of (load "....") will allow for the
> loadee to provide macros to the loader.
> 
> One problem is that the program, as its currently written, makes
> significant use of the run-time nature of load in order to configure
> the program state.
> 
> If I do switch to using (include "...") then I run into problems with
> the macros, as in loadee, where the macro blah makes use of
> process-names.  Which again (as I understand it), at macro expand time
> process-names will not be defined, and so this causes problems.
> 
> It feels frustratingly close that I can run the program with the
> compiler off, but not with it on.  I suspect there's still some way to
> go in order to make things really guile 2/3 ready.  So, is it possible
> for me to force the compiler off from within the program itself?  This
> would allow me to make a first step from 1.8.8 to 2/3 (compiler off),
> and then work on moving to compiler on after that.
> 
> Finally, I assume the model for how guile loads, expands, executes
> changed between v1 and v2.  Is there any specific hints/tips for how
> to make the transition?
> 
> Any help and advice offered would be gratefully received.
> 
> Thanks,
> Andrew
> 
> [1] https://www.gnu.org/software/guile/manual/guile.html#Local-Inclusion
>