Re: Blockers for Guile 2.2

[Jonas Hahnfeld]

Am Samstag, dem 19.02.2022 um 19:58 +0100 schrieb Jean Abou Samra:
> Le 19/02/2022 à 17:57, Jonas Hahnfeld a écrit :
> > Hi all,
> > 
> > I'd like to discuss what are considered blocker issues for a switch to
> > Guile 2.2.
> > 
> > After the release of 2.23.6, there were reports of major problems on
> > Windows, namely that the binaries were broken when extracted with the
> > Windows Explorer (#6281) and that file names with special characters
> > didn't work (#6282). I think I found solutions for both of them, either
> > already merged (!1194 for #6281) or in review (!1219 for #6282).
> > 
> > The second large topic was performance of the binaries with Guile 2.2,
> > which we know will be worse without compiled bytecode. In
> > https://lists.gnu.org/archive/html/lilypond-devel/2022-02/msg00099.html
> > Jean writes
> > > [Guile bytecode for LilyPond's .scm files] should be added eventually
> > before we make a full switch.
> > 
> > I don't fully agree and think that bytecode compilation shouldn't block
> > the switch. In my opinion, it would be fine for the next development
> > releases to be somewhat slower if that results in Guile 2.2 being
> > available sooner. The trade-off will be different for a stable release
> > where we certainly want LilyPond to be as fast as possible. That was
> > the main reason why I posted about GUILE_AUTO_COMPILE last year, to
> > have a fallback ready in case we can't get proper compilation working
> > (and showing that the compiled bytecode gives very similar performance
> > to Guile 1.8, of course).
> 
> 
> 
> I am not sure we are talking about the same issue. Here I was talking
> about shipping .go files with the binaries. I wonder if you
> interpreted it as getting compiled code with 'guild compile' rather
> than GUILE_AUTO_COMPILE.

No, I didn't. GUILE_AUTO_COMPILE is the only way of compiling that we
have right now.

> I think it's OK if the bytecode files are
> generated by GUILE_AUTO_COMPILE (I have been looking into making
> 'guild compile' work but don't have meaningful results to report
> yet). I don't think we must have bytecode files in the installers in
> order to retire GUB and start making releases exclusively with the new
> system. I do think we should ship bytecode in the installers before we
> make a stable release with Guile 2, or before we completely drop
> support for Guile 1 in the sense of removing it from CI and starting
> to accept code that doesn't work with it.

I find these statements contradicting. As soon was we start doing
releases with Guile 2.2, this is the only version that we must test and
that we must develop for. It doesn't make sense to have developers
working with Guile 1.8 locally if users are on Guile 2.2. I'm firmly
convinced that the order must be
1. only test with Guile 2.2 in CI
2. make configure only look for Guile 2.2 by default
3. do a release with Guile 2.2 only
4. unless emergency happens, drop all code related to Guile 1.8 very
soon after

> Having byte-compiled files in the
> user's cache removes the ability to completely uninstall and negates
> the ease of moving the installation around with static binaries.

Byte-compiled files from LilyPond never end up in the user's cache,
they always go to the versioned directory in share/ (as you write in
your other reply).

> Plus,
> the logging output with all warnings and ';;;' can look scaring for
> Joe User -- I worry that less computer-educated people are going
> to think LilyPond is doing something bad to their system.

Only if users set GUILE_AUTO_COMPILE=1 themselves, and there are far
worse environment variables to break about every program on your
computer. FWIW think it's bad to tell users about this variable (but
that harm is already done).

> There
> is also a big issue with the way Guile determines if a bytecode
> file is up-to-date. I could be wrong, but as far as I can see,
> it takes any bytecode that has a newer date than the source, which
> means that if we produce bytecode via GUILE_AUTO_COMPILE in the
> user's environment, and the user tries to downgrade LilyPond to an
> older version while keeping the install in the same location, it
> will break mysteriously because the cached .go file will still have
> a newer timestamp than the corresponding .scm file, and thus won't be
> recompiled.

(already addressed by your reply)

> Could you elaborate on the technical difficulties posed by
> shipping .go files?

You need to get them; ie the scripts need to compile files such that
all .scm files are compiled (including files like accreg.scm that need
to be loaded explicitly), and then collect all of them and "install"
them into the right place. For cross-compilation to mingw, you need to
collect them from the native Linux binaries.

signature.asc
Description: This is a digitally signed message part