Re: how to get left hand side symbol in action

[Christian Schoenebeck]

On Dienstag, 9. Juni 2020 07:58:56 CEST Akim Demaille wrote:
> Hi Christian,
> 
> > Le 8 juin 2020 à 13:32, Christian Schoenebeck <schoenebeck@crudebyte.com>
> > a écrit :> 
> > On Montag, 8. Juni 2020 07:46:27 CEST Akim Demaille wrote:
> >> YYTERROR was not a documented feature.  I'm curious: how did you use it?
> > 
> > I need it for the auto completion feature that we discussed before and
> > which I am using as parser injected code in some projects here and there,
> > e.g.:
> > 
> > http://svn.linuxsampler.org/cgi-bin/viewvc.cgi/linuxsampler/trunk/src/netw
> > ork/lscp.y?revision=3054&view=markup#l1605
> Ah, I see.  All this should be replaced by Bison's native support now.

I must say I had no time yet to look at this new Bison feature thoroughly, 
last glimpse I had on it though gave me the impression that it would not be 
able to replace that code.

For instance in that linked example I was not using a scanner at all, but 
instead pseudo non-terminals actually being terminals e.g.:

CREATE  :  'C''R''E''A''T''E'
        ;

So in that use case the next expected token should be returned as "CREATE", 
not just "C". So my code looks at the individual grammar rules to decide 
whether the corresponding symbol should be interpreted as terminal or non-
terminal accordingly.

Then the other thing is that my auto completion code actually auto completes 
as much as possible, not just the very next token. So if you have a highly 
redundant language (like it is often the case with human readable protocols), 
then it would auto complete several tokens up to the point where an actual 
input decision would have to be made. For instance, say you were writing an 
SQL parser and the user typed ('^' illustrating the cursor position):

        CREATE TABLE foo (id bigint UNIQUE U^

then it would auto complete it to:

        CREATE TABLE foo (id bigint UNIQUE USING INDEX TABLESPACE ^

So it would have added 3 tokens (not just one), spaces between them and space 
past the last one, where it finally would need to stop as the next token would 
be a user specified name (i.e. a user decision has to be made).

This overall feature makes things more complicated, as you have to track all 
previous parser states to detect & prevent endless recursions while trying to 
walk and fill the tree along all next possible productions.

> My question was really about the base requirement.  But I see no %require at
> all, not even say %require "3.0".

Correct, because it is supposed to work with all commonly spread Bison 
versions out there. So it supports ancient Bison versions (>=20 years?) up to 
latest 3.6.x [now ;-)]. At least so far I have not received any report about 
compilation issues due to some unsupported ancient Bison version (or any in 
between).

And BTW this was also the 1st compilation issue with a recent Bison version in 
more than 6 years. That's quite good I would say, so also thanks for taking 
care of not breaking things! :)

> I'm curious: why don't you require a modern Bison, *and ship the generated
> files*?  Waiting for the end users to have installed recent versions of the
> generators does not buy you a lot of freedom, and forces you to uglify your
> parser.

Reason for still supporting Bison 2: the license. Remember I also use this for 
commercial projects.

About shipping pregenerated parsers: I already do, for release tarballs that 
is (not for development versions though, which these reports were about).

Actually most reports about parser related compilation errors were always 
about users compiling a pregenerated one (i.e. release tarballs), in which 
case regenerating the parser with Bison always fixed the issue for them.

Best regards,
Christian Schoenebeck