Re: Rename variant and lex_symbols options

[Joel E. Denny]

On Mon, 13 Feb 2012, Akim Demaille wrote:

> Hi Joel!  (or whoever reads this message!)

Hi Akim.

> I hope I'm not infringing too much on your user
> space :) but I'd like your opinion on the proper
> names to give the %define lex_symbol and %define
> variant.  They must be renamed for 2.6 to be
> releasable.

Thanks for asking.

> We've already had a similar discussion a couple of
> years ago, but I can't find the details, and don't
> remember exactly the point we reached.

I didn't find it either.

> * variant
> The point of "variant" is to allow objects (not pointers
> to objects) to be used to type the symbols in the C++
> LR parser.
> 
> So we have, for instance :
> 
> > %token <::std::string> TEXT;
> > %token <int> NUMBER;
> > 
> > list:
> >   /* nothing */ { /* Generates an empty string list */ }
> > | list item     { std::swap ($$, $1); $$.push_back ($2); }
> > ;
> 
> It does have an influence on the API, since yylval can no
> longer be used "simply".  So it could be something like
> api.symbols.variant, or api.values.variant...

api seems reasonable given that it does affect the generated API exposed 
to the scanner, at least.

I believe we had come to the conclusion that we should avoid Boolean 
variables from now on.  The rationale was that we so often outgrow 
true|false with some other possibility we didn't originally think of.  
Could we have api.value = union|variant?  Also, notice the use of singular 
as discussed below.

> * lex_symbols
> The point here is to provide an API to build the symbols
> in such a way that it is not possible to return a semantic
> value incompatible with the token kind (e.g.,
> [0-9]+   yylval.sval = yytext; return INTEGER;):
> 
> instead you write:
> 
> > [0-9]+   return yy::parser::make_INTEGER(text_to_int (yytext), loc);
> > [a-z]+   return yy::parser::make_IDENTIFIER(yytext, loc);
> > ":"      return yy::parser::make_COLON(loc);
> 
> Again, it has an influence on the API, so may
> api.tokens.constructors (we already have api.tokens.prefix
> which probably should have been api.token.prefix),

This is my fault.  My logic was to use plural when there's more than one 
of something.  However, I now see that always using singular is probably a 
simpler rule to remember and not really so misleading as I thought.  For 
example, lr.default-reduction, lr.keep-unreachable-state, and 
api.token.prefix would all have been fine.

> or api.token.object.
> 
> Both are meant to be used together.  Maybe actually I should
> enforce this so that there are less combinations to check.

So, lex_symbols can't be used without variant?  Is it possible that might 
ever change?  Sorry, I haven't studied the details.

If lex_symbols will only ever make sense with variant, then maybe we need 
to extend the api.value enum.  variant-constructor?  I'm not sure.

If that doesn't seem right, I think api.token.constructor is fine.  But 
again, should we avoid the Boolean so it can grow if necessary?  Maybe 
none|variant, which would at least make it clearer that it's a companion 
for api.variant.