%destructor documentation not quite right...

[Steve Murphy]

Hello--


I read the manual entry (for the first time) on the %destructor
directive, and thought, "Wow! Fantastic! Now I can get rid of the
'syntax err == leaked memory' problem!!!!

But I encountered some bumps in the road!

If no syntax errs occur, I already have mechanisms to free the 
structs that yyparse allocated, AFTER THE APP IS FINISHED USING THEM.

But, I'm getting calls to yydestruct even when there are no errors.

with gdb:

Breakpoint 1, yydestruct (yymsg=0xb7708 "Cleanup: popping", yytype=43,
yyvaluep=0xffbeed34, yylocationp=0xffbee0b8) at ael.tab.c:1299
1299      if (!yymsg)
(gdb) where
#0  yydestruct (yymsg=0xb7708 "Cleanup: popping", yytype=43,
yyvaluep=0xffbeed34, yylocationp=0xffbee0b8) at ael.tab.c:1299
#1  0x00065ae0 in ael_yyparse (parseio=0xf1778) at ael.tab.c:3021
#2  0x00074f3c in ael2_parse (filename=0xffbef3a8 "./extensions.ael",
errors=0xffbef3dc) at ael.flex:621
#3  0x0009cba8 in pbx_load_module () at pbx_ael.c:3436
#4  0x0009d1b4 in load_module (mod=0x0) at pbx_ael.c:3517
#5  0x00076294 in main (argc=3, argv=0xffbef544) at ael_main.c:259
(gdb)


I've included the .info text for the bison manual below... I do not see
it clearly stated below that just about ALL tokens will be "discarded"
during the parsing process, and those which are discarded because of
errors will have a yymsg argument to yydestruct of "Error:
discarding.../popping", and those which are discarded because of who-
knows-what-other-reason (maybe via reduce?) begin with "Cleanup:
discarding.../popping". So, if I make my code look like this:

%destructor {  if (yymsg[0] != 'C') free_value($$); }  start expr TOKEN
                                            TOK_COND TOK_COLONCOLON ...

Then things work a little better... No, they work a LOT better!

Even worse, the text below seems to imply that yydestruct will only be
called in association with error processing, yet the code seems to
indicate that the calls for "Cleanup" purposes is not a typo, but
carefully designed to allow the user (me) the option of freeing any or
all memory as the parser processes it. I'm not complaining that
yydestruct is called in this manner, only that the documentation hasn't
been updated to indicate this behavior, and provide relevant examples!





Quoted from the bison-2.1 info manual:
=========================================================================
3.7.6 Freeing Discarded Symbols
-------------------------------

Some symbols can be discarded by the parser.  During error recovery
(*note Error Recovery::), symbols already pushed on the stack and
tokens coming from the rest of the file are discarded until the parser
falls on its feet.  If the parser runs out of memory, all the symbols
on the stack must be discarded.  Even if the parser succeeds, it must
discard the start symbol.

   When discarded symbols convey heap based information, this memory is
lost.  While this behavior can be tolerable for batch parsers, such as
in traditional compilers, it is unacceptable for programs like shells
or protocol implementations that may parse and execute indefinitely.

   The `%destructor' directive defines code that is called when a
symbol is discarded.

 -- Directive: %destructor { CODE } SYMBOLS
     Invoke CODE whenever the parser discards one of the SYMBOLS.
     Within CODE, `$$' designates the semantic value associated with
     the discarded symbol.  The additional parser parameters are also
     available (*note The Parser Function `yyparse': Parser Function.).

     *Warning:* as of Bison 2.1, this feature is still experimental, as
     there has not been enough user feedback.  In particular, the
     syntax might still change.

   For instance:

     %union
     {
       char *string;
     }
     %token <string> STRING
     %type  <string> string
     %destructor { free ($$); } STRING string

guarantees that when a `STRING' or a `string' is discarded, its
associated memory will be freed.

   Note that in the future, Bison might also consider that right hand
side members that are not mentioned in the action can be destroyed.  For
instance, in:

     comment: "/*" STRING "*/";

the parser is entitled to destroy the semantic value of the `string'.
Of course, this will not apply to the default action; compare:

     typeless: string;  // $$ = $1 does not apply; $1 is destroyed.
     typefull: string;  // $$ = $1 applies, $1 is not destroyed.


   "Discarded symbols" are the following:

   * stacked symbols popped during the first phase of error recovery,

   * incoming terminals during the second phase of error recovery,

   * the current look-ahead and the entire stack when the parser aborts
     (either via an explicit call to `YYABORT', or as a consequence of
     a failed error recovery or of memory exhaustion), and

   * the start symbol, when the parser succeeds.


=========================================================================

I've been thru some initial testing, and the use of the %destructor
directive in the way I outlined above has (so far) eliminated all my
leaked memory problems when errors are encountered!!

Many thanks for the upgrade (which I've not noticed for years, I
guess!), and you might want to fix the documentation.

murf