Re: sticking custom meta informations to grammar rules

[Hans Aberg]

On 30 Jan 2014, at 15:15, Christian Schoenebeck <address@hidden> wrote:

> On Wednesday 29 January 2014 23:11:28 Hans Aberg wrote:
>> It seems that your should use LR(1); see the Bison manual, sec. 5.8.1, "LR
>> Table Construction". Bison by default uses LALR(1), which merges the
>> states in such a way that when an error occurs, though no further tokens
>> will be read, some additional reductions may be applied before an error is
>> issued. 
> 
> Hmm, so far this was not the case. But I keep it mind, in case this problem 
> occurs. Thanks!
> 
>> Using LR(1) guarantees that that will not happen; so you can check
>> the lookahead tokens in the state, and that is also the set of valid
>> completions. I do not know if LR(1) works with C++, though. And you may
>> need to write some code in order to find the lookahead tokens; 
> 
> I already wrote that code, that is not the problem. My current code not only 
> resolves the next expected tokens, but also already reduces them (if 
> appropriate) to the more user friendly non-terminal symbols according to the 
> grammar definition, depending on the state stack, position in the tree, etc. 
> So it already walks up the possible tree branches and decides what to return 
> as expected "possibilities" to the user.

I recall some others in the past that have written on interactive parsers 
showing completions. So it may be of interest integrating it into Bison.

> So that is already working fine. However I want to fine tune it a bit more, 
> for example I want to make it context dependent whether to reduce an expected 
> possible non-terminal to a higher non-terminal. Because sometimes that would 
> be useful for the user and sometimes it would be counter-productive for the 
> user.
> 
> So that's why I was asking whether it is possible to associate custom 
> (static) 
> informations to individual grammar rules.

The parser algorithms throw away all grammar information; Bison only keeps in 
its table information about the states. So that is not possible. If one has 
context dependencies, that is handled in the parser actions, interacting with 
the lexer, if needed.

>> you might
>> want having this code integrated into the Bison distribution, so that the
>> feature does not break with updates.
> 
> As far as I can see it, the semantics of the yyfoo tables generated by Bison 
> have been constant over many, many years. At least I got that impression 
> after 
> reading the ChangeLog and comparing Bison versions. So I am not expecting my 
> custom code (which is directly reading and interpreting the Bison generated 
> yyfoo tables) to break with future Bison versions that soon.

They are not officially a part of a public API, and may change without notice. 
Some things have changed, for example, the C++ has a different one from the C 
one.

Hans