Re: Interactive continuation prompting

[Christian Schoenebeck]

On Wednesday 25 June 2014 23:00:10 Hans Aberg wrote:
> If you want your interpreter to read and interact token-by-token, then you
> should use the push parser. If you further want to show correct token
> completions, then an LR(1) parser should be used: the Bison default is
> LALR(1), which has states compacted in such a away that though the parsing
> is always correct, there may be some actions issued before one arrives at
> the state with the correct token completion. Further, the token
> information in the .output file is not explicit in the parsing tables, so
> that must be extracted somehow.

I implemented a shell application with tab auto complete, suggestions, colored 
syntax incomplete/complete/error indication while typing, on top of a Bison 
generated LALR(1) skeleton parser. So it is possible, but like Hans said, it 
is more complicated than with other types of parser implementations, since you 
have to walk along the parse tree, deal with shifts and reductions, etc.

If anyone is interested, I could write an article with demo source code, 
describing how to implement things like this with Bison generated skeleton 
(LALR(1)) parsers. But it could take a bit, maybe in autumn, since I have a 
huge work load in current summer period to fight with ATM.

In general I am big "fan" of parser generators. I am using Bison for many 
years now and it saved me a lot of time. So thanks for that! However its 
current design is probably not appropriate anymore for todays requirements. 
Especially these two aspects:

        1. Strong separation between "lexer" and "parser" has its historical 
           reasons, but it makes things often quite problematic. There are ways 
           to deal with that of course, but it makes the .y /.l files (and 
C/C++ 
           ontop of the parser) often hard to read. Which conflicts with todays 
           tendency, that software shall be maintainable by people without a 
           computer science degree as well. So IMO those 2 tasks should rather 
be 
           combined into one single parser generator. This way, both ends would 
           also have better knowledge how to fulfill their tasks more 
           efficiently and less error prone.

        2. Bison's current design is still more focused on keeping the parser
           tables small (and the amount of tables), instead of providing 
           convenient and detailed access to the higher level informations of 
the 
           parse states, i.e. for the tasks you described. Being able to access 
           those informations conveniently at runtime, is far more important 
           today than saving some kB of application size.

Best regards,
Christian Schoenebeck