Re: [Texmacs-dev] Darcy's plan for GNU TeXmacs 2.1.1

[Massimiliano Gubinelli]

Dear all,
 some other remarks about possible schemes for TeXmacs.


- A problem with Guile 3 (and some other options) is that they use proper 
unicode strings and this clashes with our liberal use of strings as byte 
buffers. I had to do some hacky stuff to get things going in the impl.

- I think our scheme codebase does not really depend on any specific 
implementation and we could try to keep it compatible with Guile 1.6, Guile 
1.8, S7 and Guile 3 by havign all the specific code in some (one?) file and 
keeping all the other generic. Pervasive changes are required for Guile 3 (due 
to the fact that definitions are not possible in conditional code) but the 
added code could be made no-op with macros in the other implementations.

- s7 seems to me a promising option. I think we could discuss with the main 
developer to solve the issues we see: better symbol lookup in some situations 
and better memory management. 


Best
Max



> On 30. Nov 2021, at 23:35, Massimiliano Gubinelli <m.gubinelli@gmail.com> 
> wrote:
> 
> Hi all,
> 
> thanks for taking up the work on S7. I make some short remarks:
> 
> 1) please be aware that S7 is constantly updated by its creator, I usually 
> keep in sync with the upstream here 
> 
> https://cm-gitlab.stanford.edu/bil/s7  
> 
> 2) The version in TeXmacs has just a small patch in the symbol resolution 
> routine which otherwise is too slow for TeXmacs. 
> 
> 3) The creator is very responsive to any input, I think if we patch S7 that 
> we try to send our feedback upstream so that we do not diverge too much from 
> the reference implementation
> 
> 
> 4) At some point I wanted to try also another scheme: femtolisp
> 
> https://github.com/JeffBezanson/femtolisp
> 
> But I still haven't found the time to intergrate it. 
> 
> 5) One problem with Guile 3.0 is that it is not supported on Windows, so in 
> case we go for it we will have to patch it in order to compile. What I do not 
> like in Guile is that it is too large for our need, we essentially would like 
> that all I/O or OS interaction is done via TeXmacs (which then goes via Qt5) 
> so we should not have compatibility problems in the Qt layer.
> 
> 
> Best
> Max
> 
> 
> 
>> On 25. Nov 2021, at 16:41, Joy Yang Qiping via Texmacs-dev 
>> <texmacs-dev@gnu.org> wrote:
>> 
>> Hi Joris,
>> 
>>> About plans for Scheme.  After the benchmarks, I got somewhat confused.
>>> 
>>> If S7 were systematically faster than Guile (at least than Guile 1.8),
>>> then I think that moving to S7 is the simplest option for us.
>>> Otherwise, we should keep playing with Guile 3 too.
>>> 
>>> If my understanding is correct, then the main problem is the LaTeX export.
>>> Does anyone understand why this is so much slower with S7?
>>> 
>>> Another decision we could make is that we wish to keep the support
>>> for various versions of Scheme.  But then we would need a branch that
>>> works for both S7 and Guile 3.
>> 
>> 
>> Moving to S7 may not be as simple as we would like. Here is another issue 
>> that troubles me. Over my own usage experience, TeXmacs under S7 could 
>> consume more memory than I would like (over 1GB for single document editing).
>> 
>> I think S7 will benefit from a more sophisticated gc implementation. From 
>> what I have seen through the code, they are using the same memory pool for 
>> every scheme object, which isn’t ideal for TeXmacs. Since TeXmacs not only 
>> has a linear growth of persistent object, e.g., edit history for instance, 
>> during editing, but also has a lot of side computations that claims some 
>> memory for a short duration. This means the gc will often get triggered and 
>> are slower and TeXmacs will benefit from having several levels of memory 
>> pools (from persistent to short-lived) to adapt to different memory usage 
>> pattern. The memory pool grows exponentially. I have tried to tune its 
>> exponential parameter from 2 to 1.2~1.5. But from what I have observed, it 
>> only delays the inevitable. The current S7’s gc implementation cannot 
>> guarantee long-term stability of TeXmacs — memory consumption will grow to a 
>> very high level. I don’t think it is a good idea to go through with S7 as 
>> is. I could try to turn that into a linear growth, which should work fine 
>> for TeXmacs in most cases (I’m guessing). But it really isn’t meant to be a 
>> general purpose scheme implementation with those changes. We could also find 
>> ways to reduce those persistent memory throughout editing. I might also try 
>> to look for some parameters within TeXmacs that affect those (how many 
>> editing history to keep for instance) and see if it helps.
>> 
>> I think it would be better to keep both S7 and Guile 3 working for now. 
>> Right now the main problem with Guile 3 is Windows support. Neither is the 
>> perfect solution for us :(
>> 
>> Best regards,
>> Joy
>> 
>> 
>> 
>> _______________________________________________
>> Texmacs-dev mailing list
>> Texmacs-dev@gnu.org
>> https://lists.gnu.org/mailman/listinfo/texmacs-dev
>