Re: Lexical feedback and lookahead

[Evan Lavelle]

Hans Aberg wrote:
> On 19 Jul 2005, at 17:01, Evan Lavelle wrote:
> > The only way that I know a new function is coming up is that an  
> > existing function has just completed: there's no convenient keyword  
> > to give me warning. [If my grammar really was this simple, then I  
> > could probably use another token to recognise the end of a  function, 
> > but the real input is pretty complex].
>
>
> It seems me that you know that a function has been completed when the  
> function body top level "{" ... "}" has completed. This can be kept  
> track of by a bracket depth count in the lexer.

Unfortunately, the real grammar is much more complex and counting 
brackets normally isn't adequate. Besides, if I could do this, it would 
be easier just to use start states in the lexer: I need StartStates++, 
which is lexical feedback from the parser.

> > Any ideas? It seems to me that I need an 'unget' implementation.
> In Bison, it is called %glr.

They don't do the same thing. Feedback, if it worked, could be used as a 
general solution to eliminate keywords. GLR/unlimited lookahead could, 
in principle, do the same thing, but it's much less general, much less 
efficient, and it's more difficult to design the language in the first 
place. Consider this pseudo-grammar:

top : keywords_top ;
keywords_top : lots_of_keywords | NAME ;
keyword1 : more_keywords | NAME ;
keyword2 : even_more_keywords | NAME ;
...

How do you do this in GLR? It's trivial with working feedback, and you 
can recycle keywords at different levels of the grammar.

Evan