"Eating" comments: not with Flex but with Bison

[Frans Englich]

Hello,

In some languages there are constructs which are insignificant to the parse 
tree in the same way as white space (sometimes) is. Comments is one such 
example.

The Flex manual have an example on how to do it at the scanner level. Åatterns 
which matches the comments, but doesn't return tokens:

http://www.gnu.org/software/flex/manual/html_chapter/flex_11.html#SEC11

I think I have a special scenario wrt. to comment handling: in one version of 
my language, comments are allowed while in another it is not. Hence, 
depending on version I want to flag the existence of comments as syntax 
errors, regardless of whether they are valid.

I would prefer to do this at the Bison/Parser level because it is convenient: 
I have access to various information passed to the parse function, the 
YYERROR macro, and the error function.

The problem I see if I let Flex return a COMMENT token and add a non-terminal 
in the Bison grammar to implement the checking, is how to make it play well 
with the other rules -- the token gets in the way.

What would solve my problem(AFAICT) is if I could write a 
non-terminal("Comment") that matched the COMMENT token and then simply ate 
it, such that the parser could continue to deduce the "real" tokens, as if 
the COMMENT had never existed(while the action code nevertheless did the 
check whether the comment was allowed). AFAICT, something like that must be 
done, since I can't add the COMMENT token everywhere(it can appear between 
every token).

Any ideas how to do that? (something with yyclearin..?) Or I am perhaps trying 
to solve the problem in a wrong way? (perhaps I should put the handling in 
the scanner, for example)


Also, I've ask a lot of questions -- tell if I'm asking too much, or point me 
to docs if I haven't RTFM.


Cheers,

                Frans