Re: Reasons why a LilyPond-to-MEI conversion should be developed

[Urs Liska]

Hi Simon,

thank you for that comment that is very much to the point in a number of
respects. Anyway, I have a few clarifications and/or additions to report.

Am 24.10.2015 um 01:39 schrieb Simon Albrecht:
> On 23.10.2015 17:44, Urs Liska wrote:
>> Hello devs,
>>
>> I would like to get some feedback for use when preparing the mei2ly
>> application. I will deliberately not say what I think about the topic to
>> get less influenced opinions.
>>
>> We will have to define a scope for the project that is sufficiently big
>> and at the same time not too small.
> 
> That’s tautological the way you wrote it. 

Of course, sorry.

> Do I get you right
> interpreting this as ‘should be ambitious but realistic’?

Yes, that's what I wanted to write.

> 
>>   Apart from what we may be interested
>> in it's important to be plausible and credible in order to get the
>> application accepted.
>>
>> mei2ly, the possibility to use LilyPond to engrave MEI encoded editions,
>> is clearly the foundation of the project, so there's nothing to argue
>> about that. There are technical aspects, e.g. if LilyPond should consume
>> MEI
> 
> Interesting thought. I should be surprised if MEI were to consent in
> granting LilyPond this honour (as which I’d consider it). Given the
> ‘universal’ intent of MEI, they might not want to ‘take sides’ with
> LilyPond (as opposed to other typesetting software) in such a complete
> and definite way.

I'm not sure if Richard really got you right, and I'm not sure if *I*
understand you right.

The idea is adding LilyPond as a "rendering engine" for MEI editions,
making it possible to produce professional engravings from MEI editions.
If you consider this an honor then yes, they *are* consenting to that idea.

So far there is *no* option to produce high-quality engravings from MEI
(unbelievable, isn't it?). Existing editions do one of the following:
- use Verovio (http://verovio.org), a very intersting tool that can
  render MEI as SVG, more or less instantly, with automatic reflow in
  a browser, and with tools to rewrite modifications in the SVG to
  the XML source file.
  It's a great tool but not targeted at professional engraving.
  This is in line with digital editions focusing on the flexible
  *use* and manipulation of the data.
- Create specialized engraving tools for special (e.g. medieval)
  notations
- Create an "authoritative" version of the score separately
- Work with pre-existing (publisher's) engravings and only add the
  "digital" content besides that.

Adding LilyPond to the toolkit would finally open up a way to produce
professional scores from within the MEI realm. Therefore they are
extremely happy with my advances.
They are not "taking sides" but welcome an additional tool in their toolbox.

> 
>>   or if a converter should produce LilyPond input (or to have both),
>> but the conversion direction itself is of course a prerequisite of all
>> discussion.

With regard to Richard's comment I'd like to add here:
I don't think consuming MEI should be built into LilyPond's executable
because that would unnecessarily bloat it. And I don't think and
wouldn't propose that MEI becomes LilyPond's native input langugage.

With "consuming directly" I'm referring to *some* way of doing the
process without touching LilyPond's input language. For example a
secondary script might convert MEI to some format that can be fed into
LilyPond, something similar to (make-music) expressions.

This script could either run standalone and then invoke Lilypond on the
intermediate data. Or it could be implemented in a way that it runs from
*inside* LilyPond, but not from the executable but some kind of Scheme
module.

In either case this may be easier because the whole area of LilyPond
syntax could be circumvented.
One crucial point here would be the improvement of the edition-engraver,
in a way that all necessary overrides for perfecting the output could be
injected into that stream. This is also one of the points where our
project will have direct impact on LilyPond in general, providing
new/improved solutions that any LilyPond user can benefit from.

>>
>> The other way round is less clear. Should it be possible to convert
>> LilyPond scores into MEI data?
> 
> From the code samples I’ve seen, I can’t seriously imagine anybody
> _entering_ music in .mei directly.

There are people/projects who actually do that.
I wouldn't want to do so either but that's a matter of perspective. A
typical Finale user would say the very same of a LilyPond input file.

> So there must be some kind of ‘frontend’ for the input. 

There are various attempts to develop generic, semigraphical tools for
that. But the common approach so far is entering the main content in
Finale and using a path through MusicXML export and then several stages
of XSLT processing. I find this extremely non-satisfying.

Then there are a few approach of developing "native" MEI editors, but I
don't think anything really usable is available already. But one should
expect this to become available *someday*, and this is another reason to
ask why a ly2mei converter should be available *from the MEI perspective*.

A third approach is a Sibelius2MEI exporter that is currently in beta
stage. As this is done not by the Sibelius team but from some of the
core MEI members I expect this to become pretty usable. The point is
still that Sibelius won't ever be able to maintain semantic data in a
way like LilyPond. But it will definitely take out quite some pressure
for people who may be happy with a convenient input channel.

> And I’d argue
> that .ly is a serious candidate for this because it compromises between
> a syntax apt for reading and writing by humans, and the ‘structural
> integrity’ and semantic appropriation which is necessary for a project
> like MEI.

This is indeed one of my major points. I *think* .ly offers an option to
work towards a two-way solution that can be maintained with both sides
in place. The Finale or Sibelius approaches mentioned above may seem
more convenient, but I'm quite sure they will remain a one-way street:
Create content once, convert it (with more or less hassles) but then
you're stuck to the MEI domain.

> Certainly, there are downsides too: LilyPond’s input format is – through
> the Scheme interface – extensible without limit, and it’s plain also
> that there are too few definite standards on coding style. So in the
> context of this mei-ly topic we have to reconsider GLISS, which would
> have been due, yet was postponed for years now.
> And putting up stricter standards for .ly input files is a daunting
> task, since there are good reasons for the current diversity: ranging
> from a single-file melody of less than 50 lines of code, requiring only
> the most basic and simple constructs, to something like the ‘Trunknes
> Lied’ project or an oratorio edition like Nicolas Sceaux did several,
> making excellent use of a highly individuated setup involving a complex
> directory structure, using build and maintenance scripts featuring not
> only Scheme, but also Python and/or Makefile (which latter is even
> recommended in Lily’s documentation!). One must doubt if it is at all
> possible, but it’s at least most difficult to find a balance here, and a
> viable solution working for all this diversity at once… The small melody
> I mentioned would be converted to .mei with fair ease, for the largest
> projects it would surely be impossible. But neither is it possible to
> produce such works in LilyPond without a very sophisticated setup.
> The examples are extreme of course, and I imagine the Messiah would be
> split in one .mei file per movement. Yet – a lot to consider there…
> So far some thoughts.

This outlines one of the main points to discuss. I'm sure converting Das
Trunkne Lied in its current form is absolutely out of range, at least
initially.
Regarding a general-purpose ly2mei export converter we will surely have
to make compromises about what can be (initially) supported. One point
if interest will definitely be to use LilyPond itself to do as much of
the work as possible, especially "resolving" all that Scheme stuff. I
mean, without that one would even have to parse things like \transpose
"manually".
Talking about ly2mei as an "input channel" to MEI one will probably have
to define some subset of supported functionality. And for that I don't
see such a big problem with a restriction. However, this will have to be
carefully considered.

Best
Urs

PS: hope your concert went well, unfortunately I couldn't get away from
home yesterday ...

> 
> Yours, Simon


-- 
Urs Liska
www.openlilylib.org