Re: Weird issue with dotted-list-notation

[David Kastrup]

Urs Liska <address@hidden> writes:

> Well, I do have a minimal example outlining what I want to do, but
> "unfortunately" this works exactly as expected:
>
> \version  "2.19.16"
>
> testDotList =
> #(define-void-function (parser location opts path)
>    ((ly:context-mod?) list?)
>    (for-each
>     (lambda  (p)
>       (display  p)
>       (newline))
>     path))
>
>
> \testDotList  path.to.whatever
>
> \testDotList  \with  {
>   optionOne = ##t
>   optionTwo = ##f
> }
> path.to.something.else

Any string that could be a notename will not be parsed as string.  Try
quoting "path.to.something.else".  Then the predicate "list?" is a
recipe for trouble particularly in connection with optional arguments.
Try "symbol-list?" instead.

> As it is the file compiles fine, but when I comment out line 13 (the
> pointless "ly:message") the compilation results in errors starting
> with
>
> <included string>:1:1 <0>: error: unknown escaped string: `\include'

That's rather curious.  \include is recognized in the lexer with rule

<INITIAL,chords,lyrics,figures,notes>\\include           {
        yy_push_state (incl);
}

It is not even a "reserved word" but hardwired.  So it would appear that
\\include manages to seep into the lexer in a state different from the
above list.  Either that, or your "\include" is not "\include" but
contains some funny Unicode characters, like a non-breaking non-space
(or byte order mark) or something else.  I rather suspect the state,
however.

> If I prepend the music expression with a \relative clause I get yet
> other errors, starting with
>
> openlilylib/ly/scholarly/diplomatic-line-breaks.ily:3:1 <0>: error:
> syntax error, unexpected \header, expecting (backed-up?)

(backed-up?) is an internal token used by the parser.  This message
means that you managed to confuse the parser to a degree where its
internals leak out.

So you managed confusing _both_ the parser _and_ the lexer beyond their
expected scope.

> So the function behaves (breaks) differently depending on what comes
> after it. If I uncomment the ly:message line or insert e.g.  a
> #(define foo 'bar) between \useModule and the music expression or the
> markup everything works as expected again.

Try calling LilyPond with -ddebug-parser and redirecting the rather
voluminous output somewhere.  It's probably the last few screens before
the error occurs that are relevant.

> So this is the first indicator that somehow LilyPond or Guile don't
> reliably determine when the list? argument has finished.

It needs to look ahead for that: another '.' may be following.  It may
be that you managed to embed this lookahead problem into a context where
it interferes with file inclusion.

> This assumption is backed up by trying to change the dotted-list
> notation to regular Scheme list notation. If I change line 10 to
>
> \useModule #'(scholarly diplomatic-line-breaks)

Yup.  So changing list? to symbol-list?, even though advisable, will not
likely make a difference: the lookahead issue for '.' should be the
same.

-- 
David Kastrup