GOP2-5 - ly language discussions

[Graham Percival]

http://lilypond.org/~graham/gop/gop_6.html

** Summary

We’ve gone over the same arguments many times, so let’s try to
resolve them. This proposal (seen on the website or in the
lilypond-extra/gop repository) supercedes any previous emails.

Fluff will go on a new mailing lilypond-quacks mailing list.
Serious proposals, if any, will go to lilypond-devel. Anybody with
a serious proposal must be familiar with the Extending manual,
must write up a formal proposal, will be subjected to multiple
rounds of questioning, etc.


** Motivation

Before stabilizing the syntax, I think we should have a discussion
about possible changes. Many people would like to talk about the
ly "language" (regardless of whether that involves the parser,
lexer, naming of functions and keywords and pitches, etc). Whether
“possible changes” means a “1% chance” or a “0.00001% chance” is
irrelevant at present. The goal is to share ideas. If you don’t
like fluff discussions that will probably go nowhere, don’t read
those emails.

I don’t know how to make this more clear. I want to have free
discussions, with no expectations of anything being implemented.
If this doesn’t seem appealing to you, there is no need to panic.
Some people enjoy singing in choirs; other people enjoy playing in
rock bands; other people listen to electronica. There is no need
to complain about other people’s leisure activities just because
you don’t enjoy those activities.

There is some concern about users without technical knowledge
talking about the language. To those concerns, I quote

    If you want to build a ship, don’t drum up the men to gather
wood, divide the work and give orders. Instead, teach them to
yearn for the vast and endless sea.

    - Antoine de Saint-Exupery (1900–1944), “The Wisdom of the
      Sands” 

In other words, if casual discussions can draw people into
considering language changes, those language changes will
necessarily involve a technical implementation (after discussions
on lilypond-devel), and if people are excited about these changes,
they will learn how to work on the parser.


** Clarifying terms: the ly language

There’s some ambiguity in the term "syntax" (at least, some people
might understand that word in different ways. So I’m coining a new
term: "the ly language". This refers to anything that takes place
inside a ly file.


** Clarifying terms: GLISS

Another source of misunderstandings is the term "GLISS". That
acronym comes from "Grand Lilypond Input Syntax Stabilization",
which used the term "syntax" in a possibly misleading and/or
incorrect manner.

I see two separate projects here:

  - Stabilization: we very slowly and cautiously declare certain
portions of the ly language to be "not open to future changes". In
particularly, we guarantee that certain .ly input files will
compile for all of lilypond 3.x, without any convert-ly or other
modifications. This takes place in:
    https://github.com/gperciva/lilypond-extra/tree/master/gliss
    http://lilypond.org/~graham/gliss/

    The previous GOP2-3 discussion applies to this,
    http://lilypond.org/~graham/gop/gop_4.html

  - Fixing problems: some parts of the ly language are confusing
to users, while other parts are confusing to computers. Some of
this confusion (to either humans or computers) may be unavoidable,
but I’m certain that some of the confusion could be resolved.

    It would be unfortunate if we stabilized a portion of the ly
language if it contained fixable problems. To mitigate this risk,
I want to have discussions with users about what problems they
encounter and how they could be fixed. None of those discussions
will necessarily mean that those problems will be fixed,
particularly if making things simpler for humans would make it
more complicated for computers. But the first step to looking at
whether something can be fixed is to gather information about the
problems.


** Mailing list

I suggest that we have a separate mailing list to discuss wild
ideas. Initially these will probably be about modifications to the
ly language, but other candidates are mutopia, kickstarter,
crowd-typeset music, closer ties with online music editors, etc.
This mailing list will aim to have the casual atmosphere of a
friendly discussion at a pub or coffee house. To reflect the "wild
discussions" nature while maintaining a reference a pond of
lilies, I suggest the name lilypond-quacks. A more mundane
suggestion would be lilypond-casual-chat.

These discussions on lilypond-quacks are not formal proposals, and
will not be acted upon. In exchange, nobody on that email list
will complain about technically infeasible ideas, wasting
developer’s time, having to defend the parser, or anything like
that. That list will welcome all members – there will be no
expectation that people discussing ideas will be familiar with the
parser, be capable of producing patches, or even will have read
the Extended manual. The intent behind moving informal ideas to a
separate list is to avoid causing programmers any worry from
technically infeasible ideas.

If an idea on lilypond-quacks seems to be well-liked and somebody
wants it to become an actual part of the ly language, that person
should create a formal proposal (or possibly work with a number of
people to create a proposal together) and send it to
lilypond-devel. However, they should be aware of the warnings
under the “formal proposals” section.

In addition to discussing wild ideas about the ly language, this
list will also provide an opportunity to educate people about what
is possible with the existing syntax. For example, I recently
suggestion an "improvement" which could allow the use of accents
and non-ascii characters in identifiers – only to be told that
this is already possible! However, this education should follow
the above guidelines about being welcoming, not expecting people
to be familiar with technical details, etc. Sarcastic and
disparaging comments about people’s lack of knowledge will not be
welcome on this list.


** Formal proposals

If somebody has a serious suggestion for a change to the ly
language (with the exception of renaming internals, which we do on
a completely ad-hoc basis), there will be a much more involved
process. All proposals must be sent to lilypond-devel.

Ideally, this will include a patch, examples of ly files before
and after the change, at least two weeks of discussion (similar to
GOP), etc. At a very minimum, the proposal must take into account
previous relevant discussions on lilypond-devel, the Changes
documents, and the Extending manual. Any omissions or mistakes in
a formal proposal may be subjected to sarcastic and disparaging
comments.


** ly++

One vague possibility might be to split (or extend) the ly
language in a manner similar to TeX and LaTeX. This GOP proposal
does not endorse this possibility, but merely mentions it in case
we end up with vastly divergent informed opinions on the ly
language.

Namely, the current language could remain (almost?) unchanged,
while an additional layer (ly2? lz? ly++ ?) could provide an
easier way to write music, which would then be translated into ly
for normal compiling. This could resolve a great deal of friction
between people who want more “syntactic sugar” and those who want
less sugar (or at least, no more than current).

Such an extension is not intended for any additional functionality
that could be loaded like gregorian.ly or bagpipe.ly, and any
argument in favor of ly++ would need to demonstrate that it could
not be fulfilled with a .ly init file. 


- Graham