Re: Token types with constructor

[Martin Trautmann]

Hi,

its funny that 4 persons have 4 oppinions how bison should cooperate
with C++.
Wolfgang: does your description have anything to do with your anivision
parser? I couldn't find any free for strings there...

Am Mittwoch, den 01.09.2004, 22:22 +0200 schrieb Wolfgang Wieser:
> Hello Martin & Hans!
> 
> On Tuesday 31 August 2004 22:03, you wrote:
> > [...lots of text about token types...]
> >
> So I don't really see why you seem to make things so difficult...
> Especially since it is really easy to quickly call the C++ constructors and 
> destructors from an (otherwise plain C) [parser] code: 
> 
> I assume the parser allocates a stack of token elements using: 
> 
>   Tok *stack = (Tok*)malloc(N*sizeof(Tok));
> 
> Okay, so when we need a token, then let's use one of the array elements 
> but don't forget to call the constructor before. This could be done from 
> within the push (shift) operation: 
> 
>   Tok *t = new(&stack[stack_top_index]) Tok();
> 
Am I right that you want to modify yacc.c:1163 (bison 1.875)
  YYDPRINTF ((stderr, "Shifting error token, "));

  *++yyvsp = yylval;
To
  YYDPRINTF ((stderr, "Shifting error token, "));

  new(++yyvsp) YYSTYPE();
  *yyvsp = yylval;

> This requires that the token type used inside bison (i.e. not the 
> user-supplied one), has a custom operator new: 
> 
>   struct Tok
>   {
>     short yyss;
>     YYSTYPE yyvs;
>     void *operator new(unsigned int /*size*/,void *ptr)
>       {  return(ptr);  }
>   };
> 
Is this required in any case?

> And when the token is no longer used, "delete" it again, i.e. call the 
> destructor: 
> 
>   t->Tok::~Tok();
> 
Where could this be inserted? In YYPOPSTACK?

> I've been doing things like that several times and it worked well. 
> Just write two #defines which do the ugly tasks for us. For normal C 
> parsers, these #defines are no-ops (making sure that the C compiler does 
> not stumble over operator new, etc) and for C++, we use the implementations 
> above. What do you think? (It's teaching C code C++ behaviour :)
> 
According to your opinion this two defines should be inserted in the
official version of yacc.c?

So may I summon the currently discussed possibilities to use bison in a
C++ Framework:
1) Use a real C++ skeleton (Hans: how can one specify a user defined
skeleton in bison?)
2) Construct a Token in the lexer and destruct it in every rule. There
the experimental %destructor definition is needed to avoid memory leaks
during error recoverage. Bison calls this when it discards tokens.
3) Use a derived skeleton of the official yacc.c skeleton:
3a) Construct all Tokens on stack allocation. This is currently already
done for the initial Stack:
yacc.c:762:  YYSTYPE yyvsa[YYINITDEPTH];
Correct stack reallocation may be relized defining yyoverflow. But I
don't like about yyoverflow that it needs to keep track whether it
reserved some memory during an earlier call or whether it is the
original memory reserved by parse() on the stack. And there is no final
call that may be overridden to clean the reserved stack.
3b) Construct Tokens on stack push, Destruct on stack pop

problem in 3) (if not using yyoverflow):
replacing "union yyalloc" with "struct yyalloc" to allow YYSTYPE with
constructor causes a problem in YYSTACK_RELOCATE:
# define YYSTACK_RELOCATE(Stack)                                \
    do                                                          \
      {                                                         \
        YYSIZE_T yynewbytes;                                    \
        YYCOPY (&yyptr->Stack, Stack, yysize);                  \
        Stack = &yyptr->Stack;                                  \
        yynewbytes = yystacksize * sizeof (*Stack) + YYSTACK_GAP_MAXIMUM; \
        yyptr += yynewbytes / sizeof (*yyptr);                  \
      }                                                         \
    while (0)
Stack is replaced by yyss and yyvs. yyptr->Stack is used to pass a
variable with the correct type to YYCOPY. If one changes union to struct
yyptr->yyvs points not to the same adress as yyptr. This may be fixed by
increasing YYSTACK_BYTES by one YYSTACK_GAP_MAXIMUM.

In the bison documentation I just read that compiling the parser with 
C++ doesn't generate a growing stack. Is that right? I didn't find any
condition like this in the source.

bye

Martin

--