Re: Parser: make optional arguments compatible with lookahead (issue 41720043)

[dak]

Reviewers: lemzwerg,


https://codereview.appspot.com/41720043/diff/1/lily/parser.yy
File lily/parser.yy (right):

https://codereview.appspot.com/41720043/diff/1/lily/parser.yy#newcode1073
lily/parser.yy:1073: if (!unsmob_music ($$))
On 2013/12/13 06:13:32, lemzwerg wrote:
> Looks like a slightly incorrect indentation...

How so?  This file is (see first line) basically indented in the Linux
style.  It's true that this tends to place the brace on the same line as
the if.  If it is on a separate line, however, it is lined up with the
closing brace.

I certainly can put the brace in the same line, but that's not
correcting indentation, merely compressing the code a bit more.

What's more annoying is that the copy&paste passages often use spaces
instead of tabs, while the global indentation style is supposed to use
tabs.  This partly stems from a brief period where this file got edited
while project-wide directory file variables told Emacs not to use tabs.

The inconsistency is somewhat annoying, but in the interest of letting
"git  blame" work, gratuitous changes while moving lines around are not
a good idea.

Description:
Parser: make optional arguments compatible with lookahead

Basically reimplementing the MYBACKUP and MYREPARSE macros based on an
experimentally verified understanding of the LALR(1) parser produced
by Bison, this makes it possible to greatly simplify the grammar while
making optional funciton arguments (and particularly the following
non-optional arguments even when optional arguments are being skipped)
behave much more predictable.

Further simplifications are feasible: this is a first pitch.  Consists
of the following commits in reverse order:


Adapt documentation for optional arguments now that lookahead is
permissible


Parser: properly disambiguate pitches from music in function arguments

In many instances, function arguments were parsed greedily without
looking at the function argument predicates first.  This did not allow
using subsequent function arguments of type pitch and duration since
they were combined into one music expression.

Parser: remove command_element and command_event synonyms of tempo_event


Parser: remove pitch_arg and PITCH_ARG


Parser: inline a few "common" expressions occuring only once now


Remove all closed music expressions


Permit event functions to use a non-closed argument list.


Add location data to extra_tokens_ stack in lexer.


Parser: let MYREPARSE and MYBACKUP back up even in presence of lookahead

This implies that the lookahead token must not yet have made an impact
on the state stack other than causing the reduction of the current
rule, or switching the lookahead token while Bison is mulling it over
will cause trouble.

Since an LALR(1) works on the top of the stack, this can usually be
guaranteed.  The "Too much lookahead" message is not produced.  If
these constructs are misused, the parser should typically complain
about the replacement token being unexpected.

The advantage is the ability to vastly simplify syntax rules that up
to now have to avoid lookahead tokens with a lot of drudgework.

Move the extra_token mechanism out of the lexer proper

Calling lex->pop_extra_token explicitly in the parser's yylex function
makes things considerably more robust.  It also allows pushing and
popping the EOF condition, necessary for backing up right at the end
of a file, like with ##{ \relative { c d e f } #} where the music
expression is checked against being a pitch, and then is backed up.

Please review this at https://codereview.appspot.com/41720043/

Affected files (+258, -449 lines):
  M Documentation/extending/programming-interface.itely
  M lily/include/lily-lexer.hh
  M lily/lexer.ll
  M lily/lily-parser.cc
  M lily/parser.yy