Re: Blockers for Guile 2.2

[Jean Abou Samra]

Le 19/02/2022 à 17:57, Jonas Hahnfeld a écrit :
> Hi all,
>
> I'd like to discuss what are considered blocker issues for a switch to
> Guile 2.2.
>
> After the release of 2.23.6, there were reports of major problems on
> Windows, namely that the binaries were broken when extracted with the
> Windows Explorer (#6281) and that file names with special characters
> didn't work (#6282). I think I found solutions for both of them, either
> already merged (!1194 for #6281) or in review (!1219 for #6282).
>
> The second large topic was performance of the binaries with Guile 2.2,
> which we know will be worse without compiled bytecode. In
> https://lists.gnu.org/archive/html/lilypond-devel/2022-02/msg00099.html
> Jean writes
> > [Guile bytecode for LilyPond's .scm files] should be added eventually
> before we make a full switch.
>
> I don't fully agree and think that bytecode compilation shouldn't block
> the switch. In my opinion, it would be fine for the next development
> releases to be somewhat slower if that results in Guile 2.2 being
> available sooner. The trade-off will be different for a stable release
> where we certainly want LilyPond to be as fast as possible. That was
> the main reason why I posted about GUILE_AUTO_COMPILE last year, to
> have a fallback ready in case we can't get proper compilation working
> (and showing that the compiled bytecode gives very similar performance
> to Guile 1.8, of course).



I am not sure we are talking about the same issue. Here I was talking
about shipping .go files with the binaries. I wonder if you
interpreted it as getting compiled code with 'guild compile' rather
than GUILE_AUTO_COMPILE. I think it's OK if the bytecode files are
generated by GUILE_AUTO_COMPILE (I have been looking into making
'guild compile' work but don't have meaningful results to report
yet). I don't think we must have bytecode files in the installers in
order to retire GUB and start making releases exclusively with the new
system. I do think we should ship bytecode in the installers before we
make a stable release with Guile 2, or before we completely drop
support for Guile 1 in the sense of removing it from CI and starting
to accept code that doesn't work with it. Having byte-compiled files in the
user's cache removes the ability to completely uninstall and negates
the ease of moving the installation around with static binaries. Plus,
the logging output with all warnings and ';;;' can look scaring for
Joe User -- I worry that less computer-educated people are going
to think LilyPond is doing something bad to their system. There
is also a big issue with the way Guile determines if a bytecode
file is up-to-date. I could be wrong, but as far as I can see,
it takes any bytecode that has a newer date than the source, which
means that if we produce bytecode via GUILE_AUTO_COMPILE in the
user's environment, and the user tries to downgrade LilyPond to an
older version while keeping the install in the same location, it
will break mysteriously because the cached .go file will still have
a newer timestamp than the corresponding .scm file, and thus won't be
recompiled.

Could you elaborate on the technical difficulties posed by
shipping .go files?



> What do you think? Are there are other issues that need to be taken
> into account?


Performance *with* bytecode. I am thinking to the messages
from Thomas Scharkowski where he reports LilyPond being twice
as slow as normal even with it. (I will soon write another email
to ask him to take some debugging steps so we can hopefully
understand if something in particular is causing this slowdown,
like Fontconfig regenerating its cache every time or such).


Thanks,
Jean