Re: Anybody else with an interest in parser wrangling?

[Jean Abou Samra]

Le dimanche 19 mars 2023 à 17:51 +0100, David Kastrup a écrit :  
>  
> I've not been particularly active in the last years, and there has not  
> really been a significant pickup in activity concerning syntax/parser.  
> Now for better or worse, before I picked up tenure there was GLISS, the  
> "Grand Lilypond Input Syntax Something" that sort of tried a top-down  
> prescription of what syntax would be desirable.  
>  
> This suffered from a lack of feedback and input, suggestions and  
> inspiration from the technical/implementation side and led, among other  
> things, to a lot of churn between myself and the maintainer at that  
> time, Graham, that ultimately contributed to him releasing the reins of  
> the project because he figured he wasn't helping.  
>  
> His organisational talents that fleshed out roles and procedures working  
> reasonably well with our scattered resources and interests of developers  
> and documenters and other contributors can still be witnessed in the  
> LilyPond project's somewhat unique organisational approach for getting  
> contributions integrated and, to the degree possible, vetted.  
>  
> But while my desire for work on user-pointing and internal design and  
> architecture at that time sort of unfolded mostly in a vacuum, the years  
> since then have not seen a lot of uptake.  
>  
> For example,  
>  
> git shortlog -n -s --since '10 years ago' lily/parser.yy  
>  
> is sort of one-sided (admittedly, with '1 year ago' this looks  
> different, though removing the -s argument in order to see the details  
> also makes clear that a lot of work was not syntax related, with not  
> much more than one minor and one more elaborate addition).  
>  
> As an example, I am just wrangling with extending user-definable  
> functions in the parser, and end up with things like reduce/reduce  
> conflicts that need to get ironed out.  Bison options like  
> -Wcounterexamples by now help with visualising what goes wrong (in my  
> time, I used an Emacs Lisp contraption to come up with conflict  
> examples), but one still needs to come up with plans how to  
> circumnavigate such obstacles and it usually ends up being an exercise  
> of trial and error a lot.  
>  
> There also is a lot of potential for making ping-pong progress.  I  
> realize that I am not exactly the most fun person to be working with,  
> but also I tend to get stuck on boring or repetitive tasks to an  
> inordinate degree.  Of course it is not an inviting process to tackle  
> those, but someone being slowed down to 20% of their potential will  
> still easily beat someone being slowed down to 0.2% of their potential  
> regardless of how impressive or not the respective potentials may look  
> on paper, or more exactly before doing the gritty work of actually  
> getting down on paper.  
>  
> So how to better involve others?  The parser may be one of those areas  
> with an awful amount of shoestring and glue, namely fiddling around  
> until things happen to work.  All that fiddling happens in private  
> before commits end up in master, meaning that it has no opportunity to  
> end up contagious the way it happens now.  
>  
> That's not really fabulous regarding the "bus factor" in that area.

I would feel a lot more comfortable with modifying the parser if there was an 
explanation, in code comments or in the CG, of how the parser/lexer interplay 
works, when lookahead is OK or bad, and how to avoid it when necessary. Things 
like the comment above MYBACKUP

```
// The following are somewhat precarious constructs as they may change
// the value of the lookahead token.  That implies that the lookahead
// token must not yet have made an impact on the state stack other
// than causing the reduction of the current rule, or switching the
// lookahead token while Bison is mulling it over will cause trouble.
```

are obscure to me.

signature.asc
Description: This is a digitally signed message part