Re: Trailing comments in bison

[Christian Schoenebeck]

On Sonntag, 6. Dezember 2020 17:24:40 CET Maury Markowitz wrote:
> In my flex I have these rules:
> 
> REM.*       { yylval.s = strndup(yytext + 3, yyleng - 3); return REM; }
> '.*         { yylval.s = strndup(yytext + 1, yyleng - 1); return QUOTEREM; }
> 
> And in my bison I formerly had this:
> 
> statements:
>     statement
>     {
>         $$ = g_list_prepend(NULL, $1);
>     }
> 
>     statement ':' statements
>     {
>         $$ = g_list_prepend($3, $1);
>     }

Yes, that LGTM for Altair Basic which I guess you are referring to.

> So it seemed obvious that I just needed:
> 
> 
>     statement "'" statements
>     {
>       $$ = g_list_prepend($3, $1);
>     }

No, that's a dead end. You rather want that to be handled on lexer side.

> But that doesn't work, the scanner sees the "QUOTEREM" token and parses out
> the statement properly, but that eats the quote so it no longer matches
> this pattern. So that leaves it with the REM being part of the original
> statement, which then fails as a syntax error (rightfully so). I could
> rewind and uneat the quote, but that seems like the wrong solution. Or I
> could not parse the quote in flex and only look for them in the bison, but
> then I'd have to have a bunch of rules depending on whether it's the start
> of the line or inside it or has a colon, etc.
> 
> I assume this is not an uncommon issue, what's the canonical solution?

What's your intention with comments, do you really need them on grammar side 
for something? If yes, example purpose? Usually comments are just dropped on 
lexer side, in which case you don't "return" anything on lexer side, and that 
would probably alrady fix the problem you have, without adjusting anything on 
grammar side that is.

Best regards,
Christian Schoenebeck