Re: lilypond progress

[Thomas Morley]

Hi Arne,

I really apreciate your interest and the energy you've put on LilyPond.
Though I beg to differ in some regards.
Please excuse if my wordings are a little rough, I'm a non-native speaker..

2017-03-12 18:37 GMT+01:00 Arne Babenhauserheide <address@hidden>:
> Hi,
>
>
> With all this focus on problems, I’d like to give a perspective on
> progress with Lilypond:
>
> In the past 6 years Guile 2.x for Lilypond changed from "it does not
> work at all and Lilypond might be purged from distros" to "it’s a lot
> slower but it fully works"

Not sure yet about "fully working", but I compiled lilypond with
guile-2.1.8 and got a successful make, a full make doc and the
regression-tested ran as well without noticeable problems.
That's nice ofcourse.

Though, problems will be more visible with larger scores. (All
snippets in the docs and in the reg-tests are fairly small)
Thus I tried a huge file:

lilypond-2.19.57 (self-compiled with guile 1.8.8)
real    9m17.934s
user    6m25.912s
sys        0m9.188s

lilypond-2.19.57 (self-compiled with guile 2.1.8)
first run, success:
real    45m27.500s
user    61m35.300s
sys        0m7.376s

second run, error:
Backtrace:
          16 (apply-smob/1 #<catch-closure 230caa0>)
In ice-9/eval.scm:
   293:34 15 (_ #(#(#<directory (lily) 23a5000>) #<variable 26baba…>))
    619:8 14 (_ #(#(#(#(#(#(#(#<directory (lily) …>) …) …) …) …) …) …))
In srfi/srfi-1.scm:
    640:9 13 (for-each #<procedure 29e4560 at ice-9/eval.scm:333:13…> …)
In ice-9/eval.scm:
    619:8 12 (_ #(#(#(#(#(#<directory (lily) 23a5000> # …) …) …) …) …))
In ice-9/boot-9.scm:
    834:9 11 (catch _ _ #<procedure 3802460 at ice-9/eval.scm:386:1…> …)
In unknown file:
          10 (ly:parse-file "Urtext.ly")
           9 (ly:book-process #<Book> #< Output_def> #< Output_def> #)
           8 (ly:multi-measure-rest::set-spacing-rods #<Grob MultiMe…>)
           7 (ly:separation-item::calc-skylines #<Grob NonMusicalPap…>)
           6 (ly:hara-kiri-group-spanner::pure-height #<Grob Vertic…> …)
           5 (ly:axis-group-interface::adjacent-pure-heights #<Grob …>)
           4 (ly:axis-group-interface::pure-height #<Grob NoteColum…> …)
           3 (ly:grob::stencil-height #<Grob NoteHead >)
           2 (ly:note-head::print #<Grob NoteHead >)
In ice-9/eval.scm:
    159:9  1 (_ #(#(#<directory (lily) 23a5000>) #<Grob NoteHead >))
In unknown file:
           0 (ly:duration-log ())

ERROR: In procedure ly:duration-log:
ERROR: In procedure ly:duration-log: Wrong type argument in position 1
(expecting Duration): ()

third run, success:
real    55m38.747s
user    75m27.116s
sys    0m8.900s

So this looks not exactly stable.

And we still may be dropped from major distros or lilypond with guile2
is offered, which may give us bad reputation.

>
> The path forward changed from "get Lilypond on Guile 2.x to work at all
> or we go down" to "speed up Lilypond on Guile 2.x".
>
> And the risk changed from "Lilypond on Guile 2.x might be infeasible and
> all users might lose Lilypond" to "Lilypond might regress to the speed
> it had 3 years ago¹ which might temporarily render it too slow for some
> users".
>
> ¹: Factor 4 is just 36 months of hardware development.

Let's look at some values.
I tested a medium file of my own (it's not that easy to find something
which compiles with all versions of the last decade)

GUILE 1.8.7
lilypond 2.12.3 from 2009
real    0m42.330s
user    0m40.856s
sys    0m1.508s

lilypond 2.14.2 from 2011
real    0m38.657s
user    0m34.428s
sys    0m3.272s

lilypond 2.16.2 from 2013
real    0m39.587s
user    0m35.436s
sys    0m3.536s

lilypond 2.18.2 from 2014
real    0m56.183s
user    0m47.220s
sys    0m8.484s

lilypond 2.19.56 from 2017
real    0m27.877s
user    0m24.232s
sys    0m0.524s

GUILE 1.8.8
lilypond 2.19.57 selfcompiled during past days
real    0m26.829s
user    0m26.488s
sys    0m0.380s

GUILE 2.0.14
lilypond 2.19.57 selfcompiled during past days
real    1m5.555s
user    1m15.732s
sys    0m0.592s

GUILE 2.1.8
lilypond 2.19.57 selfcompiled during past days
real    0m59.971s
user    1m12.332s
sys    0m0.612s

2.18.2 is not that nice but all other lily-versions will beat
lily-guile2-versions by far.

Additionally let me say, without being able to give a proof, for small
and medium ly-scores I've got the impression lily-guile-2.1.8 is
faster than with guile-2.0.14 and 2.1.7 and the gap to lily with
guile-1.8.8 is not that wide anymore.
But sometimes not and for unknown reasons, but mostly if extended
user-scheme-code is present in the ly-file, which feeds my concerns
about stability.
For each huge scores I tested, the gap is still huge.

>
> All that with the promise that the right kind of optimizations might
> actually make Lilypond on Guile 2.x significantly faster than on Guile
> 1.8.

Well, would be nice.

>
> TLDR: That’s great progress! The future of Lilypond is already
>       secure. It might become even more awesome than Lilypond on 1.8.
>
> I know it might not always look like that, but take a step back: Wow!
>
> Kudos to all involved and all tackling it today!
>
>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken

Best,
  Harm