Re: Inserting extra tokens

[Erik Sandberg]

On Friday 18 August 2006 09:24, Hans Aberg wrote:
> On 17 Aug 2006, at 08:51, Erik Sandberg wrote:
>
> There is, in general, no good way to insert a token, as the LALR(1)
> algorithm that Bison uses to create the parser may or may not need a
> lookahead token in each parsing position (i.e., a set of rules, each
> of with a dot in it, as in the states of the .output file that Bison
> can write). Thus, one does not know what tokens have been read when
> the parser is in a particular state and its parsing position.

My idea was to check dynamically whether 0 or 1 lookahead tokens has been 
read, and take different actions depending on this (either queue a token in 
the lexer, or manipulate the current lookahead token). Would this fail?

If so, do you know about any other parser generator that supports insertion of 
tokens?

> > 3. The argument list uses a grammar similar to
> > arglist: argument COMMA arglist | argument SEMICOLON ;
> > (an argument can be a complex expression)
>
> Now, I do not see exactly how this precedence problem relates to
> yours, as you have a different syntax. But if you only admit a
> limited number of arities, you could list them all in the .y grammar.

We currently use something similar, I'm trying to find a more generic 
solution.

> Otherwise, you will have to use the other method indicated above:
> create a dynamic arguments list object, and then use the token arity
> to work it out after the parsing of the rule.

This might be feasible, but I'm not completely sure: There are problems with 
typechecking, as there are different types (MARKUP, MUSIC) of arguments. If a 
sequence of MUSIC is expected somewhere, then FUNCTION MUSIC MARKUP MUSIC 
should give an error if FUNCTION is unary, but not necessary if it's binary. 
Of course it's possible to work around this, but I fear that lots of logic 
will have to be moved out from the parser.

Thanks for your help!
-- 
Erik