Re: Parsing from continuous stream

[Bob Rossi]

On Tue, Jan 22, 2008 at 05:00:54PM +0000, Daniel Kraft wrote:
> Bob Rossi wrote:
>>>>> start_symbol:
>>>>>   real_start_symbol
>>>>>   {
>>>>>     storeValueForProcession($1);
>>>>>     YYACCEPT;
>>>>>   }
>>>>> ;
> >>
>>> Thanks for the hint, it looks neat as far as I have seen yet!  But I'm 
>>> not yet sure how this will help me exactly; will yypush_parse return 
>>> without this YYPUSHMORE whenever it encounters the start symbol (or any 
>>> symbol?), so I can process it in my code and continue calling 
>>> yypush_parse afterwards, with the state including all look-ahead 
>>> conserved?
>> So basically, the idea is that you would pass in another structure to
>> the parsing function, and when the grammar recognized that a rule was 
>> matched from a particular token, it could set the output paramater. This
>> shoudl be thread safe as well.
>
> This sounds like the way I went as above (store the semantic value, and use 
> YYACCEPT to return from the parser; but I had problems with this approach 
> as the YYACCEPT obviously threw away read look-ahead tokens which were 
> missing the next time I called the parser (I did change my grammar, which 
> was possible without much problems, so it needed no look-ahead for the 
> start-symbol; now it works satisfyingly).
>
>>> BTW, how does one bootstrap the autotools stuff for bison?  Trying myself 
>>> with aclocal; automake -a; autoheader; autoconf (like this) gives me 
>>> errors in the resulting configure related to gl_* symbols; I suspect that 
>>> autoconf is missing some m4-files or the like?
>> Well, I've never attempted to do it manually. I just run the script
>> 'bootstrap'. I haven't done this in a few months, so, hopefully it's
>> still the same.
>
> Thanks, I'll try it with this one.

Great, please let us know your success or failure stories. This new
feature will be in the next release, and your feedback is highly
appreciated. 

One other issue, I thought about another possible way you could solve
your problem without using the push parser approach. However, it's
slightly more hacky. You could preparse commands the user is typing, for
instance, capture everything up until the '\n' character. Once you know
they typed a command, you can create an in memory buffer with flex, and
provide that to bison to parse. It will parse it and you will get back
a data structure. You could do this over and over.

I however didn't like that design as it seems kludgy to me. It was 
reasonably difficult for me to figure out what the "end of the command"
was for the language I was parsing. That was my motivation for the push 
parser.

Bob Rossi