Re: [GLISS] turn xxx.yyy into ("xxx" "yyy")

[David Kastrup]

Warming up a previous discussion because of new insights/ongoing work.

David Kastrup <address@hidden> writes:

> Jan Nieuwenhuizen <address@hidden> writes:
>
>> Werner LEMBERG writes:
>>
>>>> if we write xxx in LilyPond, this is considered to be a string.  I
>>>> want xxx.yyy.zzz to be a list of strings ("xxx" "yyy" "zzz").
>>>
>>> Go!
>>
>> isn't using symbols
>>
>>    '(xxx yyy zzz)
>>
>> more scheme-like, schemonic, schemesque?
>
> With LilyPond, we already have taken the choice
>
> Name -> "Name"
>
> and I would consider it rather awkward then if we had
>
> Name.other -> (list 'Name 'other)
>
> If we are going to choose the symbol representation (which comes at the
> cost of interning, a cost that is consolidated if you can do enough
> eq?-like comparisons that are more interesting for scalars), it does not
> make sense that we use strings for scalars.

It turns out that logical fallacies have turned the idea of using
Name.other generously as a function argument moot.  Here is the key
element from the crucial regtest note-names.ly:

  \language "italiano"
  sol4 fa mib re

Do you see the problem?  If "italiano" could be followed by . "bello",
the parser needs to check a lookahead token before it can decide whether
the function argument is complete.  It checks the next token, which
happens to be the STRING "sol" rather than '.', decides that it is not
part of the function argument, and calls the music function \language.
This music function switches the note language to Italian and proceeds
to reexamine STRING "sol" which unfortunately should have been
NOTENAME_PITCH g instead, but the lexer has already done its job.

So if we want to avoid this kind of fallacy, there are a few ways out.
I decided to take a reasonably safe route by foregoing lookahead for '.'
unless explicitly told so.  How does a function tell LilyPond to look
for a string sequence like that in a function argument?  Of course,
using the predicate.  The function sees a STRING token with an
associated string value `val' and it checks whether the predicate would
be fine with accepting val.  If so, the string gets accepted and
LilyPond does not look further.

The interesting case occurs when (pred? val) would return #f.  In that
case, LilyPond checks whether (pred? (list val)) would be true, and only
_then_ it tries looking for a dot-separated sequence of strings,
including the single-string case.

One rather sobering consequence is that any command accepting a grob
specification will _not_ be able to take a proper string generated in
Scheme using #... for it.  It will always require at least a _list_ of
strings.  This is consistent with 2.16 behavior of \override/\revert etc
where you had to at least use $... to get a string into this place (it
is not consistent with the current more lenient 2.17 behavior, but it is
not likely anybody noticed so far).

Now this opens an interesting bikeshed.  If we are going to special-case
dotted string lists including the single-member case, there is no
fundamental difference to calling (pred? (list (string->symbol val)))
rather than (pred? (list val)).

So using xxx.xxx as an entry method for #'(xxx xxx) rather than
#'("xxx" "xxx") becomes feasible: we need a separate entry method
anyway, strings will not get accepted when using the non-polymorphic
#xxx form, and $xxx will change itself to whatever form we accept, like
normal strings would.

Using the symbol list form would have the advantage that

    \override TextSpanner #'(bound-details left text) = "rit."

could equivalently be expressed as

    \override TextSpanner bound-details.left.text = "rit."

and

    \override Bottom.TextSpanner #'(bound-details left text) = "rit."

as

    \override Bottom.TextSpanner bound-details.left.text = "rit."

or even

    \override #'(Bottom TextSpanner) bound-details.left.text = "rit."

Astute readers will notice that this resembles decimal notation, with
the exception that the central pivoting point is marked by an _absence_
of a dot rather than its presence.

One question worth considering is whether in general it would make sense
to _also_ accept scalar symbols in Scheme arguments rather than only
lists of symbols.  While we can't accept scalar strings for dotted lists
without compromising the lookahead implications of string arguments,
autoconverting a string argument to a single symbol would not seem to
suffer too many drawbacks.

Back to the original mail with some more updated info:

> There is one advantage, though: after
>
>     xxx = a . b
>
> \xxx will not get classified as a MARKUPLIST_IDENTIFIER.  However, I
> have come to the conclusion that both MARKUP_IDENTIFIER as well as
> MARKUPLIST_IDENTIFIER are a bad idea.

And as of

commit b0b7a5a1b794edcec21e59417aaac0f89441120e
Author: David Kastrup <address@hidden>
Date:   Fri Sep 14 07:53:10 2012 +0200

    Eliminate MARKUP_IDENTIFIER and MARKUPLIST_IDENTIFIER
    
    Those and their LYRIC variants are no longer generated by the lexer.
    Instead, SCM_IDENTIFIER takes over their function.  One advantage is
    that standard Scheme strings and lists of strings (including the empty
    list!) are no longer receiving special syntactic status when used as
    identifier.

they are history.

> Also after
>
> xxx = #'()
>
> \xxx is a MARKUPLIST_IDENTIFIER which is plain silly.  Yes, even after I
> remove the classifications of MARKUP_IDENTIFIER and
> MARKUPLIST_IDENTIFIER, you will be able to use
>
> xxx = a . b
>
> \xxx
>
> at top level, with it being equivalent to \markuplist { a b } and I am
> not all too enthused about that.

Converting to a symbol list would avoid that consequence.

So to draw to a conclusion: this is one example where the fine points of
creating an extensible programming and syntactic model
upwards-compatible with the preexisting hardwired ad-hoc semantics in a
manner where one feels everything fits together in a sensible manner is
something sufficiently involved that it does not make sense writing
whitepapers in advance.  The details affecting an implementation evolve
only while one is actually working on and with the code.

In other words: I am taking a bit longer than initially planned to do
this feature in a manner I consider fit for integration.  On the plus
side, I am somewhat confident that people will tend to agree the results
integrate in a meaningful manner into LilyPond.

-- 
David Kastrup