Re: Token types with constructor

[Martin Trautmann]

Hallo Hans,

the mail below didn't come over the list as well. Did I cause your
reply-all button to work impropperly?
Is it possible to write questions to the list without being subscribed
to the list?

bye

Martin

e.g.: I consider this as a private discussion

Am Mittwoch, den 01.09.2004, 19:04 +0200 schrieb Hans Aberg:
> At 22:03 +0200 2004/08/31, Martin Trautmann wrote:
> >I think I understood the main problem. But can you tell me what a bison
> >parser may do with tokens? Am I right with:
> >1) flex creates the token object (no problem: constructor is called)
> >2) bison stores it in a stack *++yyvsp = yylval;
> >Both variables are of type YYSTYPE, so operator=() is called
>
> Right.
>
> >3) when a stack overflow occurs: bison copies the stack
> >(!!! no copy Constructor / operator= is called !!!)
>
> The C-stack just makes a memcpy(), so under C++, no user copy constructors
> are invoked. (Strictly speaking, POD's have a built in, default copy
> constructor which both C and C++ invoke.) The get that, one needs to use a
> proper C++ stack, such as std::deque, std::vector or something.
>
> >4) bison calls a rule function that gets a stack element as parameter
> >This access is done using the correct type and shouldn't do any harm.
>
> There should be no access problems, if only the object exists (which it may
> not do, if not copied properly).
>
> >And there is one version of YYCOPY:
> >      do                                     \
> >     {                                       \
> >       register YYSIZE_T yyi;                \
> >       for (yyi = 0; yyi < (Count); yyi++)   \
> >         (To)[yyi] = (From)[yyi];            \
> >     }                                       \
> >      while (0)
> >That would have no problems using operator=() as well. So the only
> >additional thing to do would be to force the usage of that.
> >
> >But I think in my small parser with the token type I showed you, stack
> >overflows didn't occur during my tests.
>
> You have to ensure that it does not. Somebody else will have to reply how
> to do that, as I am currently happy with my C++ parser that does that
> automatically. Essentially, with the current setup, and if you are using
> nontrivial copy constructors with the C-parser, you need to ensure that the
> stack is fixed in size, and that the parser aborts when the stack overflows.
>
> >And in my other parser I do a special version of reference counting:
> >"object there or not there":
> >class Token
> >{
> >  int type;
> >  u{
> >    T1* t1;
> >    T2* t2;
> >     ...
> >  };
> >  T1& t1();
> >  T2& t2();
> >     ...
> >  Token() : type(no_token) {}
> >};
> >
> >The only problem is a memory leak, that I don't free any Tokens finally
> >on the stack.
>
> When such an object is memcpy()'ied, you will get two object pointing at
> the same object without the refcount being increased. So eventually, you
> will end up with an object that will be deleted twice -- a segfault.
>
> >Oh, Finally I found a real problem:
> >Token::operator=() can't be sure whether the destination object was
> >already initialized by a constructor or not.
> >
> >So there are to ways:
> >1) YYSTACK_ALLOC constructs all tokens on the stack
> >2) back to the roots: how does plain C work? ;-)
> >
> >Do you know any parser that stores more than just some integers and is
> >implemented in a clean way?
>
> If you by "more than just some integers" mean a C++ non-POD, then I  gave
> example of that -- my parser, used with refcount objects. It is posted in
> the Bison-Patches list with instructions on how to use it.
>
>   Hans Aberg
>
>
--