Re: lilypond progress

[David Kastrup]

Thomas Morley <address@hidden> writes:

> Hi Arne,
>
> I really apreciate your interest and the energy you've put on LilyPond.
> Though I beg to differ in some regards.
> Please excuse if my wordings are a little rough, I'm a non-native speaker..
>
> 2017-03-12 18:37 GMT+01:00 Arne Babenhauserheide <address@hidden>:
>> Hi,
>>
>>
>> With all this focus on problems, I’d like to give a perspective on
>> progress with Lilypond:
>>
>> In the past 6 years Guile 2.x for Lilypond changed from "it does not
>> work at all and Lilypond might be purged from distros" to "it’s a lot
>> slower but it fully works"
>
> Not sure yet about "fully working", but I compiled lilypond with
> guile-2.1.8 and got a successful make, a full make doc and the
> regression-tested ran as well without noticeable problems.

Oh.  I thought we still had encoding problems, so that's good news to
me.

> Though, problems will be more visible with larger scores. (All
> snippets in the docs and in the reg-tests are fairly small)
> Thus I tried a huge file:
>
> lilypond-2.19.57 (self-compiled with guile 1.8.8)
> real    9m17.934s
> user    6m25.912s
> sys        0m9.188s
>
> lilypond-2.19.57 (self-compiled with guile 2.1.8)
> first run, success:
> real    45m27.500s
> user    61m35.300s
> sys        0m7.376s
>
> second run, error:
> Backtrace:
>           16 (apply-smob/1 #<catch-closure 230caa0>)
> In ice-9/eval.scm:
>    293:34 15 (_ #(#(#<directory (lily) 23a5000>) #<variable 26baba…>))
>     619:8 14 (_ #(#(#(#(#(#(#(#<directory (lily) …>) …) …) …) …) …) …))
> In srfi/srfi-1.scm:
>     640:9 13 (for-each #<procedure 29e4560 at ice-9/eval.scm:333:13…> …)
> In ice-9/eval.scm:
>     619:8 12 (_ #(#(#(#(#(#<directory (lily) 23a5000> # …) …) …) …) …))
> In ice-9/boot-9.scm:
>     834:9 11 (catch _ _ #<procedure 3802460 at ice-9/eval.scm:386:1…> …)
> In unknown file:
>           10 (ly:parse-file "Urtext.ly")
>            9 (ly:book-process #<Book> #< Output_def> #< Output_def> #)
>            8 (ly:multi-measure-rest::set-spacing-rods #<Grob MultiMe…>)
>            7 (ly:separation-item::calc-skylines #<Grob NonMusicalPap…>)
>            6 (ly:hara-kiri-group-spanner::pure-height #<Grob Vertic…> …)
>            5 (ly:axis-group-interface::adjacent-pure-heights #<Grob …>)
>            4 (ly:axis-group-interface::pure-height #<Grob NoteColum…> …)
>            3 (ly:grob::stencil-height #<Grob NoteHead >)
>            2 (ly:note-head::print #<Grob NoteHead >)
> In ice-9/eval.scm:
>     159:9  1 (_ #(#(#<directory (lily) 23a5000>) #<Grob NoteHead >))
> In unknown file:
>            0 (ly:duration-log ())
>
> ERROR: In procedure ly:duration-log:
> ERROR: In procedure ly:duration-log: Wrong type argument in position 1
> (expecting Duration): ()
>
> third run, success:
> real    55m38.747s
> user    75m27.116s
> sys    0m8.900s
>
> So this looks not exactly stable.

Ugh.  It's hard to say just who is "to blame" here since the more
asynchronous garbage collection makes it much more important that the
cautions surrounding scm_remember_upto_here* are heeded, and those were
already documented for Guile 1, and adherence in the LilyPond code base
tends to be somewhat lackadaisical.
>
> Additionally let me say, without being able to give a proof, for small
> and medium ly-scores I've got the impression lily-guile-2.1.8 is
> faster than with guile-2.0.14 and 2.1.7 and the gap to lily with
> guile-1.8.8 is not that wide anymore.
> But sometimes not and for unknown reasons, but mostly if extended
> user-scheme-code is present in the ly-file, which feeds my concerns
> about stability.
> For each huge scores I tested, the gap is still huge.
>
>>
>> All that with the promise that the right kind of optimizations might
>> actually make Lilypond on Guile 2.x significantly faster than on Guile
>> 1.8.
>
> Well, would be nice.

I'm not holding my breath.  String operations based on Unicode may be
nicer on Guile 2.  And once the compilation framework is properly used
in LilyPond, the _built-in_ (rather than user-defined) Scheme parts,
namely those in *.scm files rather than in #sexp or $sexp in *.ly files,
can be expected to run faster, making it more feasible to use Scheme
engravers and output functions and similar for material somewhat more
performance-critical than what we can reasonably do now.

But at the moment the drawbacks (including performance of interpreted
code) we take in a number of other respects don't make up for those
benefits which are rather marginal given the current code base.  Of
course that is partly due to the current code base being fit to the
shortcomings and tradeoffs of Guile 1 specifically.

-- 
David Kastrup