Re: Feature Request Idea - Server Sent Musical Scores

[hjenkins]

This sounds like an excellent feature.

Apart from ly2video, lilypondToBandVideoConverter and a third approach 
called mkvideo seem to exist.

I realize this feature might add a lot of complexity, probably in the 
way in which Lilypond renders scores, but it would be very nice to have 
a way of marking the portion of music currently playing. Say, 
highlighting the notes while they are being sounded, or having a cursor 
passing along the staff (at least hitting the end of each bar on-cue, 
maybe a ball bouncing along if it's too hard to interpolate correctly). 
This would be useful for teaching sight-reading of music, just as a 
pointer or finger is useful in learning to read text.

And this is even more complex, but there would be a lot of utility in a 
method for controlling the speed (either a manual paging feature like 
W3C Slidy, or a continuous system that counts beats from a microphone or 
conductor's baton). If you're having a music group run through a new 
piece, scrolling scores on a overhead display would save everyone from 
carrying around paper scores (which get lost and blow away and rained 
upon and so on) and music stands (which are cumbrous and fall over with 
a clatter). It would also improve attention to a conductor standing near 
the screen, and improve posture.


Given that an increasing number of illiterate people are online with 
mobiles, a simpler but parallel functionality might simply convert a 
text into a video with a synthesized voice and scrolling text. It could 
mark the future, current, and past text in three different styles and 
colours, and optionally add concurrent visual media. Even if this only 
worked in English, it would aid literacy and access to information in 
places like India.



On 2021-09-16 10:21, jgart wrote:
> Hi Goblineers,
>
> I think GNU MediaGoblin should sit down and enjoy a coffee with GNU 
> LilyPond.
>
> I wanted to share some thoughts and ask for feedback on the possibility 
> of
> adding support for musical scores?
>
> I was thinking of integrating ly2video: 
> https://github.com/aspiers/ly2video
>
> ly2video converts lilypond files into a video containing a horizontally
> scrolling musical staff which is synchronized with a MIDI-generated 
> audio
> rendering of the music.
>
> A goblineer could upload their lilypond musical score (.ly file) to 
> mediagoblin
> and mediagoblin would send it to the backend, process it with ly2video, 
> and
> return a video of their score synchronized with the MIDI and horizontal
> scrolling bar.
>
> Here are some examples of the output of ly2video in action (invidious 
> link):
>
> https://youtube.076.ne.jp/playlist?list=PLfRwjd606WZlxRU_kaUPagX3-Uv-SYRMH
>
> I imagine we could also add batch uploading of musical scores via the 
> cli as
> well as offer multiple formats of the score for download such as the 
> original
> source file (.ly), MIDI, pdf, svg, flac, ogg, etc...
>
> We could even offer musical score transposition before sending to 
> ly2video as
> well as an alternative color coded static html score view courtesy of 
> python-ly:
>
> https://github.com/frescobaldi/python-ly
>
> python-ly is already packaged in guix and is used extensively by the
> Frescobaldi
> LilyPond IDE.
>
> I think this would be an awesome feature to add and would attract 
> attention for
> GMG as I know of no other media forge that supports this.
>
> Taking this approach would also not introduce any javascript 
> dependencies since
> the musical score creation is being strictly handled in the backend and
> a video, html, etc... is produced and sent to the client.
>
> WDYT?
>
> https://invidious.io/
> https://frescobaldi.org/