Re: rfc: next guile 1.8.x release

[Massimiliano Gubinelli]

> On 30. Jan 2021, at 13:31, Ricardo Wurmus <rekado@elephly.net> wrote:
> 
> 
> Dr. Arne Babenhauserheide <arne_bab@web.de <mailto:arne_bab@web.de>> writes:
> 
>> Ludovic Courtès <ludo@gnu.org> writes:
>> 
>>> Thien-Thi Nguyen <ttn@gnuvola.org> skribis:
>>>> I would like to work (on the weekends, so as not to intefere w/
>>>> <day-job>) on preparation and release of Guile 1.8.9, targeted
>>>> for the ides of April (more or less).
>>>> 
>>>> Guile 1.8.x users: What changes do you want to see in 1.8.9?
>> 
>> I am not a Guile 1.8.x user, but changes I would like to see is anything
>> that could ease incremental porting to Guile 3.x. If there’s a way to
>> backport incompatible interfaces from 3.x so people can replace the
>> 1.8.x-specific interfaces one by one, that could make it more viable to
>> move upwards.
> 
> Guile 2 was already very different from 1.8.
> 
>>>> Guile maintainers: Any tips (process, content, interop, etc)
>>>> most welcome!
>>> 
>>> Guile 1.8 has been unmaintained for years and I wonder about the message
>>> we’d be sending by publishing a bug-fix release 11 years and 3 major
>>> versions later.
>> 
>> The message we’re sending is that we don’t leave anyone out in the cold
>> who committed to using Guile.
> 
> The mailing lists are evidence enough that users of Guile 1.8 can get
> help in migrating to version 2+.
> 
> But suggesting that Guile 1.8 is still maintained when it emphatically
> is not — that is a step too far, in my opinion.
> 

I spent the last few months of my allowed "hacking time" in trying to 
understand the situation and the possible strategies TeXmacs has wrt. extension 
language. Since the beginning TeXmacs used Guile (I think it was the natural 
choice for a GNU project). In TeXmacs Guile is an "extension language" and this 
means various things.

1) most of the codebase is in C++ and depends on other large libraries like Qt 
for cross-platform compatibility and defines its own interface to the OS
2) internally we do not use Unicode for strings
3) our code is written for a fast interpreted and dynamical scheme (like Guile 
1.8)

all these three features point to weakness of various possible solutions

1) we do not need a Scheme with a lot of OS interfaces, this duplicate 
functionalities in QT and in our own code. For example all the OS interaction 
is dealt with internal functions (also because TeXmacs has its own virtual 
filesystem). We do not need threads, GMP, etc... all these additional parts add 
bloat without providing real useful functionalities.
2) it is better for us to have a Scheme which is not picky wrt. encodings. 
3) compiled schemes or schemes without first-class modules are more difficult 
to adapt to our own flavour of scheme coding.

Currently I'm trying three different routes:

- embed S7 (which is fast, dynamic and minimal and works everywhere)
- go for Guile 3 (I've still problems with phasing of macro expansion and 
bootstrap of our codebase. Will it work on Windows? Do we really need to carry 
on all the code we do not use?)
- go for Chez Scheme (like for Guile 3 problems with compilation phases and 
bootstrap of our codebase. I'm more confident it will work on Windows and on 
new architecture, it is more compact than Guile, essentially only the compiler, 
I've still problems with the garbage collector at the level of C++ interface).

My feeling is that Scheme developers are focused on compiled static programs 
and not so much on providing a reliable extension language for other programs. 
Ideally I would agree with the idea of writing everything in a fast scheme, but 
reality is different: cross-platform GUI are written in C++, TeXmacs' 
typesetting code is in C++: nobody has the time to rewrite everything. What 
Scheme provide to TeXmacs (and I think to a project like Lilypond, which 
however I do not know well) is a way to create our own extension language where 
to specify custom behaviour: we have macros to describe the GUI, the 
keybindings, the conversion algorithms, etc... all this is a quite large 
codebase which does not fit very much in the "statically compiled module" idea. 
 Still it is a very complex layer (TeXmacs has 100.000 lines of Scheme code).

Guile 1.8 works very well for us, that is the irony... We like it a lot, it is 
fast (and we need fast because interpreted code is called at every 
keypress....) It is so good that it is not easy to find a replacement: Chibi is 
too slow, S7 is very good but I needed to patch it otherwise it had some 
problem and was much slower than Guile 1.8. 

So Guile 1.8 today remains a very good piece of software, that is the reason we 
keep it. Maybe Lilypond has similar reasons. Our problems come from the fact 
that major Linux distributions do not package it anymore, so packaging TeXmacs 
has become a pain. We are even thinking to just include the code in our 
codebase... or even include Guile 1.6 which is also good enough for us and does 
not force us to link GMP.

In my mind Guile 1.8 is a very different language than Guile 2/3 and I do not 
see reason to make difficult for people to install them in parallel as it is 
possible for Guile 2 and Guile 3. Guile 2 cannot replace Guile 1.8, in this 
respect S7 is a better replacement, actively maintained. 

What would be ideal for large programs like ours is to have an extension 
language with the following characteristics:

1) it exposes as little surface as possible to the OS (so that it is easily 
portable and adaptable),
2) allow to use other string representations, so that one can just use the 
strings which come with the application without marshalling them thru the FFI,
3)  is fast enough even with interpreted code so that interactive applications 
can rely on it.
4)  it has some consideration of interactive applications where the separations 
of code in libraries is not very appropriate: one cannot ask the user to import 
10-20 modules before being able to write some code. In TeXmacs we have a system 
where every module create procedures in a global environment. This is easy to 
do with Guile first-class modules but it is quite difficult to do in other 
schemes which are more picky about symbol lookup. 

I would be interested in hearing from other "large" adopters of Guile about 
their experience on the topics I mentioned above. 

Also I would be interested in pointers into current uses of Guile in other 
large programs so that I can learn patterns/techniques which maybe I've missed 
(how to structure the codebase, how to do cross-platform development, 
interaction with the OS, etc...)

Let me just finish by saying that at the moment we do not have clear plans on 
the future of Guile in TeXmacs, and it is probable that we will stick with 
Guile 1.8 for a while longer, so if there are patch around (e.g. to compile in 
Windows, etc...) it would be very useful to have a way to centralize them, and 
maybe to have Guile 1.8 parallel installable with Guile 2/3. 

Best,
Massimiliano

ps: some time ago I noted down some remarks about the problem of which Scheme 
to use in TeXmacs here : https://texmacs.github.io/notes/docs/scheming.html 
<https://texmacs.github.io/notes/docs/scheming.html>
comments are welcome.


> -- 
> Ricardo