Re: reentrant flex, bison glue

[John W. Millaway]

John>  %option 'reentrant-bison' causes flex to do three things:
John>
John>  1. Flex adds pointers of type YYSTYPE and (optionally) YYLTYPE
John>  to the scanner guts. The values are expected to be passed into
John>  yylex(). Flex expects these types to be defined elsewhere,
John>  (e.g., y.tab.h) and NEVER touches their values.

Akim> I don't understand this.  If Bison or Yacc is used on the top of
Akim> Flex, then the .h file is to be included, which defines these guys.
Akim> Just as you said.  So I don't understand this bit, not its
Akim> relationship with Bison.

If bison passes values to flex, the only way for these values to be
available to the user in a reentrant scanner is for the values to be
saved in the scanner state. I say "guts" but they really go in the
same place that 'yyleng' goes.  Flex preserves these values for you,
untouched, inside the scanner state (which is passed around from
functions to function).

John>  2. The macros yylval and yylloc are defined, as are the accessor
John>  functions yyget_lval(), yyset_lval(), yyget_lloc(),
John>  yyset_lloc().

Akim> Again, I don't see why these guys are there.

These are for consistancy with the reentrant flex API. The API has
get/set functions for all of its user-accessible values. I didn't
want the user to directly access the flex internals, for obvious
reasons. If you skim through the reentrant API section in the
manual, you'll see what I mean immediately.

John>  The macros can be used from within flex actions, as if they were
John>  global variables. The accessor functions allow access from
John>  anywhere (e.g., from within the parser).  The idea is that
John>  scanner action can do something like this:

John>  %% /* access yylloc as if it were global. */ \r?\n {
John>  yylloc->my_line_count++; return EOL;}

Akim> Correct, but in what way would these macros would help?


Again, it's a nicety of the reentrant API. Which would you prefer of
the two below?

// contrived example
if( yyget_text((struct yy_globals_t*)yy_globals)->text_r[0] == '\n')
    yyget_lineno((struct yy_globals_t*)yy_globals)->lineno_r++;

if( yytext[0] == '\n')
    yylineno++;

They are equivalent, but one requires the dereferencing of some kind
of scanner state (or parser state) variables EVERYTIME you want to
access those variables.  The neat thing about the second line is
that it's EXACTLY the same from the user's perspective as the
non-reentrant flex version -- the migration from non-reentrant to
reentrant should be fairly easy. This same bit of sugar is provided
for 'ltype' and 'lloc', too.



John>  3. yylex() is defined with the additional arguments YYSTYPE* and
John>  (optionally) YYLTYPE*.

Akim> So, if I understand correctly, this is meant to replace the
Akim> user-definition of YY_DECL?  How to you plan to take the case of the
Akim> parser passing additional arguments into account?  I'm referring to
Akim> #define YY_DECL yylex (...., my_struct *str)

The user can still do it by hand. This was only meant to cover the default
behavior of bison. (Or at least, the behavior described in the bison manual).

John>  The whole idea here is that a flex scanner will just "work" with
John>  a bison pure parser, without much fuss for the user.

Akim> The sole difference I can see is that YY_DECL will be defined
Akim> automatically, right?

That, and the macros, and the accessor functions, and the
incorporation of YYSTYPE* and YYLTYPE* into the scanner state.


__________________________________________________
Do You Yahoo!?
Try FREE Yahoo! Mail - the world's greatest free email!
http://mail.yahoo.com/