Re: How to make GNU Guile more successful

[Thomas Morley]

2017-03-05 0:44 GMT+01:00 Arne Babenhauserheide <address@hidden>:
>
> David Kastrup <address@hidden> writes:
>
>> Nala Ginrut <address@hidden> writes:
>>
>>> I think we have to elaborate the question clearer.
>>>
>>> 1. How to make guile-scheme more successful?
>>> I think this is similar to ask "how to make scheme more successful".
>>> This is the big question to the whole scheme community.
>>>
>>> 2. How to make guile platform more successful?
>>> I this case, let me ask a question, if we have guile-python3
>>> (compatible with most of Python3 features), and provide more
>>> convenient FFI helper function to help bind more libraries in short
>>> time, is it possible to attract more people from Python land? Given
>>> we'll have good JIT compiler in the future.
>>
>> Frankly, I doubt that migration of large Python-based applications is
>> going to be a thing when nobody can even be bothered with immersing
>> himself in the problems with migrating LilyPond from Guile-1.8 to
>> Guile-2.
>
> I worked on testing Lilypond with user installed Guile 2.x, does that
> count?
>
> Just to have it recorded somewhere, here’s a patch to lilypond along
> with a copy of the bash history of the setup (cleaned up, it was many
> times as long):
>
> ## patch
>
> From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
> Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)
>
> ---
>  configure.ac         | 7 ++++++-
>  lily/pdf-scheme.cc   | 4 ++++
>  scm/memory-trace.scm | 3 ++-
>  3 files changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/configure.ac b/configure.ac
> index d77ea15..393976b 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
>  STEPMAKE_WINDOWS
>
>  # guile executable for some scripts
> -STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
> +if test "$GUILEv2" = "yes"
> +then
> +    STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
> +else
> +    STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
> +fi
>
>  # perl for help2man and for mf2pt1.pl
>  STEPMAKE_PERL(REQUIRED)
> diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
> index da2ce2c..f5ae70c 100644
> --- a/lily/pdf-scheme.cc
> +++ b/lily/pdf-scheme.cc
> @@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, 
> "ly:encode-string-for-pdf",
>     *                  (string->utf16 str 'big)
>     */
>    if (g)
> +#if GUILEV2
> +    return scm_take_locale_stringn (g, bytes_written); // scm_take_str 
> eventually frees g!
> +#else
>      return scm_take_str (g, bytes_written); // scm_take_str eventually frees 
> g!
> +#endif
>    else
>      return str;
>  }
> diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
> index d8ffeb9..9ebd722 100644
> --- a/scm/memory-trace.scm
> +++ b/scm/memory-trace.scm
> @@ -2,7 +2,8 @@
>
>  (define-module (scm memory-trace))
>  (use-modules (lily)
> -             (ice-9 format))
> +             (ice-9 format)
> +             (ice-9 threads))
>
>  (define-public (mtrace:start-trace freq)
>    (set! usecond-interval (inexact->exact (/ 1000000 freq)))
> --
> 2.10.2
>
>
> ## bash history
>
> sudo emerge media-fonts/tex-gyre
> sudo nano /etc/portage/package.keywords/sonstiges
> sudo emerge media-fonts/tex-gyre
> sudo pmerge dblatex
> git clone git://git.sv.gnu.org/lilypond.git
> cd lilypond/
> ./autogen.sh --prefix ~/.local --enable-guile2
> mkdir -p ~/texmf/lh
> cp ~/Downloads/lhfnt35g-source.zip ~/texmf/lh/
> unzip lhfnt35g-source.zip
> cd ~/texmf/source/latex/lh
> latex lcyfonts.ins
> latex ot2fonts.ins; latex t2ccfonts.ins
> ls ~/texmf/examples/
> mkdir ~/texmf/lh
> cp *sty ~/texmf/lh/
> ./autogen.sh --prefix ~/.local --enable-guile2
> sudo emerge dev-texlive/texlive-langcyrillic
> sudo pmerge dev-texlive/texlive-langcyrillic
> make out/lilypond-invoke-editor
> cd scripts/
> /home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor 
> --no-discard-stderr
> /home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor
> ./autogen.sh --prefix ~/.local --enable-guile2
> make
> LD_LIBRARY_PATH=/home/arne/.local/lib/guile/2.2/ g++ -o out/lilypond  
> ./out/modified-font-metric.o ./out/sequential-iterator.o 
> ./out/hara-kiri-group-spanner.o ./out/completion-note-heads-engraver.o 
> ./out/slur-score-parameters.o ./out/flag.o ./out/span-arpeggio-engraver.o 
> ./out/beam-quanting.o ./out/dynamic-performer.o ./out/parse-scm.o ./out/tie.o 
> ./out/bend-engraver.o ./out/staff-spacing.o ./out/rest-collision-engraver.o 
> ./out/simple-spacer-scheme.o ./out/audio-item.o ./out/pdf-scheme.o 
> ./out/paper-score.o ./out/slur-configuration.o ./out/constrained-breaking.o 
> ./out/score-engraver.o ./out/unpure-pure-container.o 
> ./out/multi-measure-rest-engraver.o ./out/system-start-delimiter-engraver.o 
> ./out/context-property.o ./out/freetype.o ./out/interval-minefield.o 
> ./out/kievan-ligature.o ./out/completion-rest-engraver.o 
> ./out/pure-from-neighbor-engraver.o ./out/horizontal-bracket-engraver.o 
> ./out/grob-closure.o ./out/default-bar-line-engraver.o 
> ./out/control-track-performer.o ./out/file-name-map.o ./out/audio-staff.o 
> ./out/line-spanner.o ./out/grob-info.o ./out/spring.o ./out/slur.o 
> ./out/music-scheme.o ./out/program-option.o ./out/fretboard-engraver.o 
> ./out/hyphen-engraver.o ./out/page-marker.o ./out/global-vars.o 
> ./out/stream-event-scheme.o ./out/script-column-engraver.o ./out/bezier-bow.o 
> ./out/page-breaking-scheme.o ./out/lily-version.o ./out/repeat-tie-engraver.o 
> ./out/tweak-engraver.o ./out/staff-symbol-referencer.o ./out/moment-scheme.o 
> ./out/note-heads-engraver.o ./out/paper-outputter.o ./out/music-wrapper.o 
> ./out/grob.o ./out/performance-scheme.o ./out/bar-line.o 
> ./out/page-turn-engraver.o ./out/dimensions-scheme.o 
> ./out/tab-note-heads-engraver.o ./out/piano-pedal-bracket.o 
> ./out/spaceable-grob.o ./out/event-iterator.o 
> ./out/lyric-combine-music-iterator.o ./out/percent-repeat-iterator.o 
> ./out/audio-element-info.o ./out/midi-cc-announcer.o ./out/font-metric.o 
> ./out/note-spacing-engraver.o ./out/duration-scheme.o 
> ./out/accidental-placement.o ./out/music-iterator.o ./out/rhythmic-head.o 
> ./out/grob-array.o ./out/tab-tie-follow-engraver.o ./out/context.o 
> ./out/dispatcher.o ./out/scale.o ./out/chord-name-engraver.o 
> ./out/auto-change-iterator.o ./out/tie-engraver.o ./out/grid-line-interface.o 
> ./out/forbid-break-engraver.o ./out/note-column.o ./out/lookup.o 
> ./out/global-ctor.o ./out/custos-engraver.o ./out/piano-pedal-engraver.o 
> ./out/simple-music-iterator.o ./out/pitch-scheme.o 
> ./out/mensural-ligature-engraver.o ./out/articulations.o 
> ./out/horizontal-bracket.o ./out/script-column.o ./out/global-context.o 
> ./out/slur-performer.o ./out/guile-init.o ./out/stencil-scheme.o 
> ./out/font-size-engraver.o ./out/rest-engraver.o ./out/skyline.o 
> ./out/percent-repeat-item.o ./out/enclosing-bracket.o ./out/module-scheme.o 
> ./out/input.o ./out/paper-outputter-scheme.o ./out/spacing-interface.o 
> ./out/break-align-engraver.o ./out/chord-tremolo-iterator.o 
> ./out/general-scheme.o ./out/least-squares.o ./out/audio-element.o 
> ./out/paper-system.o ./out/lily-lexer-scheme.o ./out/beam.o 
> ./out/music-sequence.o ./out/music.o ./out/stem.o 
> ./out/relative-octave-check.o ./out/keyword.o ./out/translator-ctors.o 
> ./out/stencil-integral.o ./out/arpeggio-engraver.o ./out/extender-engraver.o 
> ./out/prob-scheme.o ./out/ligature-engraver.o ./out/lyric-hyphen.o 
> ./out/simple-spacer.o ./out/output-property-engraver.o 
> ./out/drum-note-performer.o ./out/fingering-column-engraver.o 
> ./out/midi-item.o ./out/undead.o ./out/repeat-acknowledge-engraver.o 
> ./out/grace-engraver.o ./out/figured-bass-position-engraver.o 
> ./out/tie-formatting-problem.o ./out/context-mod-scheme.o ./out/pfb.o 
> ./out/fingering-engraver.o ./out/profile.o ./out/break-alignment-interface.o 
> ./out/trill-spanner-engraver.o ./out/multi-measure-rest.o 
> ./out/collision-engraver.o ./out/system-start-delimiter.o 
> ./out/one-line-auto-height-breaking.o ./out/minimal-page-breaking.o 
> ./out/dot-column-engraver.o ./out/note-name-engraver.o ./out/breathing-sign.o 
> ./out/paper-book.o ./out/sustain-pedal.o ./out/tie-configuration.o 
> ./out/staff-grouper-interface.o ./out/output-def.o ./out/nested-property.o 
> ./out/timing-translator.o ./out/protected-scm.o ./out/volta-bracket.o 
> ./out/beam-performer.o ./out/beaming-pattern.o ./out/prob.o 
> ./out/part-combine-engraver.o ./out/smobs.o ./out/system.o 
> ./out/figured-bass-engraver.o ./out/paper-def.o ./out/performer-group.o 
> ./out/rhythmic-column-engraver.o ./out/volta-engraver.o 
> ./out/paper-column-engraver.o ./out/scm-hash.o 
> ./out/beam-collision-engraver.o ./out/apply-context-iterator.o 
> ./out/skyline-pair.o ./out/gdb.o ./out/spacing-spanner.o 
> ./out/stanza-number-align-engraver.o ./out/line-interface.o 
> ./out/measure-grouping-spanner.o ./out/staff-symbol-referencer-scheme.o 
> ./out/stem-engraver.o ./out/font-metric-scheme.o ./out/beam-engraver.o 
> ./out/freetype-error.o ./out/instrument-name-engraver.o 
> ./out/partial-iterator.o ./out/lily-lexer.o ./out/metronome-engraver.o 
> ./out/warn-scheme.o ./out/includable-lexer.o ./out/item.o 
> ./out/spacing-engraver.o ./out/relocate.o ./out/pointer-group-interface.o 
> ./out/measure-grouping-engraver.o ./out/grace-music.o ./out/staff-performer.o 
> ./out/slur-engraver.o ./out/function-documentation.o ./out/dots.o 
> ./out/performer.o ./out/page-spacing.o ./out/separating-line-group-engraver.o 
> ./out/axis-group-interface-scheme.o ./out/box.o 
> ./out/spacing-determine-loose-columns.o ./out/duration.o 
> ./out/stencil-expression.o ./out/phrasing-slur-engraver.o 
> ./out/side-position-interface.o ./out/pitch-interval.o ./out/accidental.o 
> ./out/pure-from-neighbor-interface.o ./out/optimal-page-breaking.o 
> ./out/spanner-break-forbid-engraver.o ./out/drum-note-engraver.o 
> ./out/stanza-number-engraver.o ./out/midi-walker.o ./out/ly-module.o 
> ./out/span-bar-stub-engraver.o ./out/book-scheme.o 
> ./out/grid-point-engraver.o ./out/scheme-engraver.o ./out/dot-column.o 
> ./out/note-head-scheme.o ./out/lily-imports.o ./out/font-config-scheme.o 
> ./out/open-type-font-scheme.o ./out/hairpin.o ./out/quote-iterator.o 
> ./out/ledger-line-engraver.o ./out/auto-beam-engraver.o ./out/pango-font.o 
> ./out/misc.o ./out/repeated-music.o ./out/figured-bass-continuation.o 
> ./out/grace-iterator.o ./out/cue-clef-engraver.o ./out/pango-select.o 
> ./out/main.o ./out/music-wrapper-iterator.o ./out/pitch-squash-engraver.o 
> ./out/tuplet-iterator.o ./out/stencil-interpret.o 
> ./out/key-signature-interface.o ./out/pitched-trill-engraver.o ./out/score.o 
> ./out/context-mod.o ./out/fingering-column.o ./out/music-function-scheme.o 
> ./out/bar-number-engraver.o ./out/break-substitution.o ./out/sources.o 
> ./out/piano-pedal-performer.o ./out/performance.o 
> ./out/volta-repeat-iterator.o ./out/slash-repeat-engraver.o 
> ./out/font-config.o ./out/lily-guile.o ./out/balloon.o 
> ./out/text-spanner-engraver.o ./out/bezier.o ./out/stencil.o ./out/clef.o 
> ./out/spacing-basic.o ./out/dynamic-engraver.o 
> ./out/time-signature-engraver.o ./out/ottava-engraver.o ./out/music-output.o 
> ./out/lily-parser.o ./out/ligature-bracket-engraver.o 
> ./out/translator-dispatch-list.o ./out/lily-parser-scheme.o 
> ./out/engraver-scheme.o ./out/dynamic-align-engraver.o 
> ./out/part-combine-iterator.o ./out/custos.o ./out/translator-scheme.o 
> ./out/slur-scoring.o ./out/accidental-engraver.o ./out/rest-collision.o 
> ./out/staff-symbol-engraver.o ./out/spanner.o ./out/note-column-scheme.o 
> ./out/piano-pedal-align-engraver.o ./out/clef-modifier.o 
> ./out/note-performer.o ./out/pango-font-scheme.o ./out/melody-spanner.o 
> ./out/tie-column.o ./out/tuplet-bracket.o ./out/episema-engraver.o 
> ./out/lyric-extender.o ./out/grob-property.o ./out/lyric-combine-music.o 
> ./out/vaticana-ligature.o ./out/cluster-engraver.o 
> ./out/instrument-switch-engraver.o ./out/part-combine-part-iterator.o 
> ./out/audio-column.o ./out/key-performer.o 
> ./out/gregorian-ligature-engraver.o ./out/context-specced-music-iterator.o 
> ./out/pango-select-scheme.o ./out/translator-group.o ./out/book.o 
> ./out/stream-event.o ./out/staff-symbol.o ./out/font-interface.o 
> ./out/lyric-engraver.o ./out/page-breaking.o ./out/engraver-group.o 
> ./out/grob-smob.o ./out/grob-array-scheme.o ./out/rod.o ./out/moment.o 
> ./out/dispatcher-scheme.o ./out/lilypond-version.o 
> ./out/tab-staff-symbol-engraver.o ./out/font-select.o 
> ./out/grob-interface-scheme.o ./out/tuplet-engraver.o 
> ./out/balloon-engraver.o ./out/template5.o ./out/paper-column.o 
> ./out/script-row-engraver.o ./out/clef-engraver.o ./out/one-page-breaking.o 
> ./out/midi-chunk.o ./out/context-def.o ./out/paper-score-scheme.o 
> ./out/axis-group-engraver.o ./out/cluster.o ./out/translator.o 
> ./out/item-scheme.o ./out/double-percent-repeat-engraver.o 
> ./out/separation-item.o ./out/key-engraver.o 
> ./out/keep-alive-together-engraver.o ./out/source-file.o 
> ./out/all-font-metrics-scheme.o ./out/self-alignment-interface.o 
> ./out/axis-group-interface.o ./out/midi-cc-performer.o ./out/input-scheme.o 
> ./out/lyric-performer.o ./out/grid-line-span-engraver.o 
> ./out/glissando-engraver.o ./out/event-chord-iterator.o ./out/chord-name.o 
> ./out/spring-smob.o ./out/score-performer.o ./out/paper-system-scheme.o 
> ./out/coherent-ligature-engraver.o ./out/context-scheme.o 
> ./out/tie-performer.o ./out/semi-tie.o ./out/chord-tremolo-engraver.o 
> ./out/pitch.o ./out/listener.o ./out/percent-repeat-engraver.o 
> ./out/note-head.o ./out/relative-octave-music.o ./out/note-spacing.o 
> ./out/dimension-cache.o ./out/spacing-loose-columns.o ./out/melody-engraver.o 
> ./out/page-spacing-result.o ./out/script-engraver.o ./out/ttf.o 
> ./out/dot-formatting-problem.o ./out/all-font-metrics.o 
> ./out/ledger-line-spanner.o ./out/staff-collecting-engraver.o 
> ./out/spacing-options.o ./out/time-signature-performer.o ./out/pfb-scheme.o 
> ./out/property-iterator.o ./out/vaticana-ligature-engraver.o 
> ./out/script-interface.o ./out/score-scheme.o ./out/one-line-page-breaking.o 
> ./out/page-marker-scheme.o ./out/spanner-scheme.o ./out/music-function.o 
> ./out/ambitus-engraver.o ./out/new-fingering-engraver.o 
> ./out/bar-check-iterator.o ./out/change-iterator.o ./out/context-handle.o 
> ./out/parenthesis-engraver.o ./out/align-interface.o 
> ./out/change-sequence-iterator.o ./out/column-x-positions.o 
> ./out/output-def-scheme.o ./out/global-context-scheme.o ./out/tie-details.o 
> ./out/rest.o ./out/note-collision.o ./out/dot-configuration.o 
> ./out/tempo-performer.o ./out/translator-group-ctors.o ./out/ottava-bracket.o 
> ./out/stem-tremolo.o ./out/mensural-ligature.o ./out/grob-pq-engraver.o 
> ./out/lily-modules.o ./out/page-turn-page-breaking.o 
> ./out/gregorian-ligature.o ./out/engraver.o ./out/rhythmic-music-iterator.o 
> ./out/mark-engraver.o ./out/page-layout-problem-scheme.o 
> ./out/vertical-align-engraver.o ./out/kievan-ligature-engraver.o 
> ./out/tie-specification.o ./out/tuplet-number.o 
> ./out/laissez-vibrer-engraver.o ./out/dots-engraver.o 
> ./out/directional-element-interface.o ./out/concurrent-hairpin-engraver.o 
> ./out/line-interface-scheme.o ./out/paper-book-scheme.o 
> ./out/pointer-group-interface-scheme.o ./out/grob-interface.o 
> ./out/open-type-font.o ./out/span-bar-engraver.o ./out/midi-stream.o 
> ./out/page-layout-problem.o ./out/input-smob.o 
> ./out/note-head-line-engraver.o ./out/bar-engraver.o ./out/text-interface.o 
> ./out/text-engraver.o ./out/grob-scheme.o ./out/arpeggio.o 
> ./out/semi-tie-column.o ./out/program-option-scheme.o 
> ./out/footnote-engraver.o ./out/simultaneous-music-iterator.o 
> ./out/breathing-sign-engraver.o ./out/grace-spacing-engraver.o ./out/lexer.o 
> ./out/parser.o ./out/../../flower/out/library.a  -ldl   
> -L/home/arne/.local/lib -L/home/arne/.local/lib/guile/2.2 -lguile-2.2 -lgc  
> -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype  
> -lfontconfig -lfreetype  -lfreetype -v 2>&1 | grep -i lib
> nm out/lilypond
> LD_LIBRARY_PATH=~/.local/lib/  out/lilypond
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:../:../scm LD_LIBRARY_PATH=~/.local/lib/  
> out/lilypond
> wget http://infinite-hands.draketo.de/infinite_hands_sheet.ly
> wget -O mf/out/emmentaler-20.otf 
> https://github.com/saebekassebil/subito/raw/master/resources/gonville/lilyfonts/otf/emmentaler-20.otf
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  
> lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
> cp lilypond-data_2.18.2-4.1_all.deb /tmp
> scp kav:/usr/share/lilypond/2.16.2/fonts/otf/*otf mf/out/
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  
> lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
> git add configure.ac lily/general-scheme.cc lily/pdf-scheme.cc 
> scm/memory-trace.scm
> time GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  
> lily/out/lilypond -I ly/ -I mf/out/ -I ps/ infinite_hands_sheet.ly
> cat 0001-Build-fixes-for-Guile-2.1.x-not-yet-functional.patch
> cat > 1 <<EOF
> diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
> index 1168ee9..2df63fc 100644
> --- a/lily/general-scheme.cc
> +++ b/lily/general-scheme.cc
> @@ -275,7 +275,8 @@ LY_DEFINE (ly_protects, "ly:protects",
>  #if SCM_MAJOR_VERSION < 2 || SCM_MAJOR_VERSION == 2 && SCM_MINOR_VERSION < 1
>    return scm_protects;
>  #else
> -  return programming_error ("ly:protects is not supported in Guile 2.1");
> +  // return programming_error ("ly:protects is not supported in Guile 2.1");
> +  return 0; // FIXME: Evil hack just to get this to build.
>  #endif
>  }
> EOF
>
> Best wishes,
> Arne
>
> PS: If this provides at least a small step towards guile 2 in lilypond,
>     it’s worth its while.
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken

Hi Arne,

many thanks for your work!
I'll have a closer look tomorrow (it's in the middle of the night here)...

The main problem is not that we can't build lilypond with 2.0.14 or 2.1.7
Checkout the branch dev/guile-v2-work in the lilypond-repository (you
will need to rebase it against current master) and you'll be able to
build LilyPond with guile-2.0.13, so I'm pretty confident it will work
with 2.0.14. For building with guile 2.1.7 an additional patch is
needed, currently I've simply deleted all about ly:protects
(brute-force, I know...).

Though the perfomance of LilyPond is dramatically slowed down. Making
it effectively unusable for any longer input.
I'm currently testing this for builds with guile 1.8.8, 2.0.14 and
2.1.7 and will report tomorrow.

Cheers,
  Harm