Re: "Eating" comments: not with Flex but with Bison

[Frans Englich]

On Tuesday 14 June 2005 11:36, Akim Demaille wrote:
> >>> "Frans" == Frans Englich <address@hidden> writes:
>  >
>  > I would prefer to do this at the Bison/Parser level because it is
>  > convenient: I have access to various information passed to the
>  > parse function,
>
> You can easily make them available to the scanner.  And in fact, you
> probably should, to have a clean, pure, interface bw the two.

Ok, not fully following here, what you mean with "make them available to the 
scanner". From what I can tell, I must in either case have patterns that 
catches the comments(perhaps that's what you mean with "make them 
available"); what I then do, if I return tokens, is another matter AFAICT.

>
>  > the YYERROR macro, and the error function.
>
> There is nothing wrong with defining a token, but not allowing it in
> the grammar.  Just return it when there is a comment (and you don't
> want them), then you'll have the expected result: a parser error.
>
>  > The problem I see if I let Flex return a COMMENT token and add a
>  > non-terminal in the Bison grammar to implement the checking, is how
>  > to make it play well with the other rules -- the token gets in the
>  > way.
>
> Of course you must not try to use the token COMMENT when comment are
> valid, just discard them.

Again a bit confused by the wording; tokens must first exist in order to be 
able to discard them, is my thinking. Perhaps you mean that "by discarding 
them" that the scanner never returns them at all.

So, to see if I've understood you correctly, you suggest that:

1. The (Bison) parser have no knowledge about COMMENT tokens(e.g no rules), 
except that it merely declares the token.

2. The (Flex) scanner has patterns that matches the comments, combined with 
that the scanner have business logic that decides whether to return a COMMENT 
token or not(and the former leading to a syntax error since the parser can't 
handle it).

Hm, then have error handling been placed in the scanner, which confuses me 
with respect to "You can easily make them available to the scanner. And in 
fact, you probably should, to have a clean, pure, interface bw the 
two."(assuming my interpretation is correct).


Cheers,

                Frans