Re: Help using bison

[Bob Rossi]

On Thu, Dec 15, 2005 at 02:08:04AM +0100, Michael Brandt wrote:
> On 15/12/05, Bob Rossi <address@hidden> wrote:
> > > The lemon parser generator uses "inverted flow-of-control", the
> > > user/lexer calls the parser for each input token.
> > >
> > > See http://www.hwaci.com/sw/lemon/
> >
> > Wow, OK, Thanks! Do you know where I would find a lexer to replace flex
> > that can do the same?
> 
> I think you don't have to replace flex at all. I was a bit inaccurate
> about the control flow. The Lexer doesn't call the parser directly.
> >From the lemon documentation:
> 
>    ParseFile(){
>       pParser = ParseAlloc( malloc );
>       while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){
>          Parse(pParser, hTokenId, sToken);
>       }
>       Parse(pParser, 0, sToken);
>       ParseFree(pParser, free );
>    }
> 
> So you just have to replace GetNextToken() with yylex, define YY_DECL
> for the extra arguments, store the return types of your flexer rules
> in hTokenID and the token value in sToken.

Yeah, I had noticed that from the documentation. However, yylex reads in
from a global FILE descriptor. I basically need a lexer that supports
something like,
  while (1) {
    c = getNextChar ();
    while (1) {
      lexer(c);
      if (lexer_found_token) {
        Parse (token, lexer_token);
      }
    }
  }

Basically, provide the lexer with data when I get it. When the lexer
finds a token, I provide it to the parser. This would be a complete
asynchronous solution to the problem.

> > I'm assuming it's impossible to change bison to act like this, short of
> > changing the entire generated parser to not be recursive descent. Is
> > that true?
> 
> Changing bison would be painfull. But i think the type of control flow
> (scanner calls parser vs. parser calls scanner) has nothing to do
> about whether the parser is top-down (recursive deschent, LL, ...) or
> bottom-up (LALR like bison).

I was assuming that a recursive decent parser needs to be recursive,
which relies upon the stack. If the generated parser relies upon the
stack then I don't think it could use the inverted flow-of-control
design.

Bob Rossi