Re: State of LilyPond with Guile 2.2

[Han-Wen Nienhuys]

On Sun, Apr 11, 2021 at 4:24 PM Jonas Hahnfeld <hahnjo@hahnjo.de> wrote:
>
> Am Sonntag, dem 11.04.2021 um 16:04 +0200 schrieb Han-Wen Nienhuys:
> > On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on
> > LilyPond development <lilypond-devel@gnu.org> wrote:
> > >
> > > All numbers are from my laptop running Arch Linux (with pango
> > > downgraded to 1:1.48.2-1 to keep out the memory hogging in 1.48.3)
> > > and
> > > measured with "/usr/bin/time -v". I use commit fce156f219 from
> > > https://gitlab.com/lilypond/lilypond/-/merge_requests/723 (see
> > > User's
> > > View) and the system-provided versions of Guile 1.8.8 and Guile
> > > 2.2.6.
> > > I don't test Guile 3.0 because it's not available as package, and
> > > there
> > > seem to be more changes needed to make it even work.
> > > I configured the build with --enable-gs-api (because I'm interested
> > > in
> > > the time spent in LilyPond, not how fast the system can fork gs)
> > >
> >
> > I assume this is also without extractpdfmark, which is also a CPU
> > hog.
>
> Correct, all numbers are without extractpdfmark.
>
> > > The first set of experiments is about things that developers care
> > > about, namely 'make test' and 'make doc'. For all runs, I used the
> > > options "-j4 CPU_COUNT=4".
> > >
> > > 'make test' when compiled with Guile 1.8.8:
> > >  * User time (seconds): 520.68
> > >  * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54
> > >
> >
> > IIRC, on my 4 CPU machine, the actual time running the regression .ly
> > files is fairly low (about 1 minute). A lot of time is spent running
> > the lilypond-book regression tests, which makes this difference all
> > the more dramatic
>
> True, but the current way of testing lilypond-book is kind of the
> worst-case scenario for LilyPond because all files are compiled
> separately. So 37 times a startup time of 2 seconds is already the
> majority of the slowdown.

This may warrant some further investigation. Ideally, the shared
snippet database makes it so we have to run lilypond only a few times.

> > >  * Maximum resident set size (kbytes): 372560
> > > 'make test' when compiled with Guile 2.2.6:
> > >  * User time (seconds): 710.35
> > >  * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04
> > >  * Maximum resident set size (kbytes): 372344
> > >
> > > -> 40% slower when using Guile 2.2.6 during development
> > >
> >
> >
> > I didn't see you mention this explicitly: for apples to apples
> > comparisons, you have to disable 'debug-gc-object-lifetimes feature.
> > It causes a significant slowdown with Guile 1.8 for make test/make
> > doc, so the comparison is probably worse.
> >
> > This comes at the expense of increased memory usage (which reduces GC
> > overhead), so you'd have to test with my GUILE patches for a
> > completely honest comparison. I think the advantage of the decreased
> > GC overhead is minor, though.
>
> Right, I didn't think about this. However, I compiled without --enable-
> checking so the actual difference is just garbage collection happening
> between the files, which you think should be minor. Also "apples to
> apples" comparison means without your patches, because that's what
> users get.

That's not what I meant. The GC between the files is the expensive
part, because it is done unconditionally and has to mark all the
reachable data, and sweep the entire heap.  Dropping that (according
to commit e36b7c7ea98) is a 20% speedup, which we already collected
for GUILE 2.2, but not for 1.8.

Due to the heap scaling bugs, removing the GC between files increases
the heap size with 1.8, and if you don't force GC, for a heap that is
5x larger, you run GC 5x fewer times. The GC runs themselves involve a
larger heap, so this evens out, but you have to mark the reachable set
5x fewer times. This is what I meant with a 'minor' advantage.

> > > In my opinion, the numbers show that we *must have* compiled
> > > bytecode
> > > for user installations in order to reach acceptable performance
> > > (memory
> > > usage seems to be fine). And while compilation by invoking LilyPond
> > > is
> > > somewhat odd, it works and would be viable for the beginning.
> > >
> > > For development, I'm less convinced. Sure, 'make test' and 'make
> > > doc'
> > > get faster but the compilation itself takes a considerable amount
> > > of
> > > time. Moreover it is my understanding the Guile is notoriously bad
> > > at
> > > determining which files to recompile, in particular when macros are
> > > involved.
> > >
> > > Personally, I would be ok with the moderate slowdown if that was
> > > the
> > > only thing preventing a hypothetic switch to Guile 2.2, and the
> > > arising
> > >
> >
> >
> > What does "switch to" mean precisely? Do you mean "remove support for
> > guile 1.8", "make 2.2 the default (when given the choice) but keep
> > supporting 1.8" ?
>
> Eventually, I think the target should be to "remove support for Guile
> 1.8". In particular once we provide binaries with Guile 2.2, that will
> be the one version we care about.

>From an engineering perspective, I like the idea of dropping 1.8,
because we can then simplify the C++ GC marking glue which is hard to
get right, but otherwise, it's still not very compelling.
Functionally, the only thing that seems really relevant is Unicode
string handling on the Scheme side.

I wonder if it isn't more practical to fork guile 1.8, and stick it
into our tree as a submodule, and always build lilypond against the
in-tree version. We'd be up for the maintenance on the 1.8 branch, but
it might well be less work than keeping up with the churn that newer
GUILE versions bring us.

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen