Re: Require delimiters to clarify context? [was: how to make decisions?]

[David Kastrup]

"Keith OHara" <address@hidden> writes:

> On Wed, 05 Sep 2012 15:47:18 -0700, Trevor Daniels
> <address@hidden> wrote:
>
>>> On Wed, 05 Sep 2012 02:54:38 -0700, Trevor Daniels
>> <address@hidden> wrote:
>>>>
>>>> There are many places in LilyPond now where delimiters are necessary
>>>> to resolve certain situations but are not generally mandatory.
>
>> Most of the examples require { .. } iff the '..' consists of more
>> than one element.
>>
>> For example
> [[  \relative c' {f f}  ]]
>
> I thought for a moment you were going to point out:
>   \new Voice {\relative c'' b b }   % Two notes
>   \new Voice {\relative {c''} b b } % Three notes
>
>> Could this behaviour be extended so braces could be used to clarify
>> command/argument boundaries in more complex cases, but with the
>> freedom to omit them (as now) if they were unnecessary?
>
> I can only see *optional* delimiters helping as a slightly-ugly way to
> let users have a new feature that would otherwise be ambiguous in some
> cases with the existing grammar.

I might point out that ##{ ... #} can serve as poor man's delimiters in
many cases.  It might make sense to eventually make { ... } do the same,
namely only cause sequential music if there are more than two elements
inside.  And possibly even allow the same sort of thing we allow inside
of ##{ ... #}.  This would make { ... } useful for delimiting most
function arguments, without changing single-event music by putting
SequentialMusic around it.

It would also bring #{ ... #} back into close correspondence with
{ ... }.

> to get 4. recognized as a duration.
>
> (I am now rather intrigued to figure how David managed to do it without some 
> other context clue, given that \midi and \layout are parallel constructions:
>   \layout{
>     \tempo 4. = 30
>     short-indent = 2.\cm }
>   \midi{
>     \tempo 4. = 30
>      short-indent = 2.\cm }
> )

Well, short-indent will still be scanned in music mode and become the
lookahead token that the parser needs for realizing \tempo is finished.
The token as such is a string.  It is turned into a variable when the
assignment has been scanned, and so while the _string_ has been
recognized in music mode, the _assignment_ happens in the context
definition.  This was one reason why it was important to me that the
definition of strings does not change across lexer mods.

> David introduced a concept "closed_music" last year, including {...}
> and other cases where the parser can distinguish its end
> unambiguously.

"unambigously" is not the problem.  "without lookahead" is the problem
regarding optional arguments, because once the parser has seen a
lookahead token, it has changed _state_, and I can't push back that
token for a different interpretation.

I am currently working on lifting that restriction.  Basically, I am
starting a new parser whenever I need to parse with secret lookahead.
That causes some trickiness in the interaction of the parsers (the
scanner is not reentrable and should keep working from a single source)
but should considerably help bringing the grammar itself back into the
range of being manageable.  It might alternatively make sense to see at
Bison's other available parser classes like GLR parsers.  However,
though it may look nice not to have to worry about the depth of
lookahead, this can play seriously foul with switching lexer modes.

> It was issue 2001, where we left him rather lonely, giving no
> comments.  Now that we have some background, we might appreciate that
> patch set.

I am still annoyed at its implications and resulting complexity and
working on ways to get around that.  Having to explain the invasive
concept of "closed expressions" in the context of music functions when
they are more a parser artifact than anything else and don't just affect
optional arguments themselves, but rather the mandatory arguments
following a _skipped_ optional argument...  Nuisance.  To be fair, I
think they started their life in the context of duration arguments, but
it seems possible to get rid of them for that purpose reasonably well by
now.

-- 
David Kastrup