[PATCH 00/16] Returning the error token from the scanner does not trigger an error message

[Akim Demaille]

The bistromathic provides quite a good example of this feature:

    int
    yylex (const char **line, YYSTYPE *yylval, YYLTYPE *yylloc)
    {
    [...]
      switch (c)
        {
        case '!': return TOK_YYUNDEF;
        default:
          yyerror (yylloc, "syntax error: invalid character: %c", c);
          return TOK_YYERRCODE;
        }
    }

Then:

    > !
    1.1: syntax error: expected end of file or - or ( or exit or number or 
function or variable before invalid token
    > #
    2.1: syntax error: invalid character: #

In both cases the parser enters the error-recovery system, but in the
second case the scanner generated a clear error message, something the
parser cannot do about lexical errors.

This feature was, so far, available in C++, using the `syntax_error`
exception.  Now all the languages can use it.

However, we must find a different name for the `error` token, as some
project already use that name expecting an error message.  It should
be noted that since `YYERRCODE` have never been documented, one should
never have depended upon it.

This is available in https://github.com/akimd/bison/pull/35 too.  I'll
push it when the CI is ok with it.

I'll probably wrap another beta soon.  There are still things TODO
though (help would be much appreciated...).

Akim Demaille (16):
  skeletons: make the warning about implementation details clearer
  style: fix a few remaining 'type' instead of 'kind'
  style: glr.c: fix indentation issue
  style: prefer b4_has_translations_if
  style: glr.c: clarify
  all: prefer YYERRCODE to YYERROR
  c++: fix a few style issues
  c++: always define symbol_name
  c++: make valid to print the empty symbol
  doc: hacking tricks
  examples: bistromathic: comment changes
  examples: bistromathic: when quitting, close the current line
  examples: bistromathic: demonstrate error recovery
  c: don't emit an error message when the scanner returns YYERRCODE
  all: don't emit an error message when the scanner returns YYERRCODE
  todo: update

 README-hacking.md                         |  14 ++
 TODO                                      | 123 +++-------------
 data/skeletons/bison.m4                   |   7 +-
 data/skeletons/c++.m4                     |   2 +-
 data/skeletons/glr.c                      |  48 ++++---
 data/skeletons/glr.cc                     |  18 +--
 data/skeletons/lalr1.cc                   | 134 +++++++++++-------
 data/skeletons/lalr1.d                    |  80 ++++++-----
 data/skeletons/lalr1.java                 |  84 ++++++-----
 data/skeletons/yacc.c                     |  34 +++--
 doc/bison.texi                            |   8 +-
 examples/c/README.md                      |   1 +
 examples/c/bistromathic/README.md         |   1 +
 examples/c/bistromathic/bistromathic.test | 164 ++++++++++++++++++++--
 examples/c/bistromathic/parse.y           |  37 +++--
 examples/test                             |  30 ++--
 src/parse-gram.c                          |  21 +--
 src/parse-gram.h                          |   5 +-
 tests/calc.at                             |  61 +++++++-
 tests/headers.at                          |   2 +-
 tests/input.at                            |   2 +-
 21 files changed, 553 insertions(+), 323 deletions(-)

-- 
2.26.2