bug#62333: 30.0.50; Issue with tree-sitter syntax tree during certain changes

[Dmitry Gutov]

On 27/03/2023 11:24, Gregory Heytings wrote:
> > > > > Again, I'm talking about using a parser library.  We may need to 
> > > > > introduce a way of limiting the parser to a certain range of buffer 
> > > > > text positions, independently of narrowing.
> > > >
> > > > Except it's already limited by narrowing.
> > >
> > > Which is a fragile, semi-broken means, as we all know.
> >
> > What is a broken mess, is user-level narrowing. And how the downstream 
> > code can never guess the purpose the narrowing was applied for.
>
> Note that this is what labeled narrowings attempts to solve.  It's 
> perhaps not the best way to deal with code that has existed for a long 
> time (because such code is not prepared to handle that kind of narrowing 
> gracefully), but I don't see why it couldn't be used in new code/modes 
> such as tree-sitter ones.  It is independent of user-level narrowing, 
> and downstream code can act differently depending on the reason for 
> which the narrowing was applied.

It indeed sounds like something we had considered before: different 
types of narrowing (at the time we only wanted to distinguish between 
two types: "soft" and "hard", the latter for use by mixed-mode features, 
or other stuff which wanted to preclude full widening, such as Info). 
But that fizzled out on the prototyping stage.

(Aside: tree-sitter itself has its own support for "ranges", which will 
work without any narrowing related stuff as long as grammars for all 
langs are available -- or perhaps the combination should be available as 
a grammar too, I'm not sure -- anyway, it doesn't need much help, but it 
would still be nice to support embedding tree-sitter managed code inside 
"regular" modes, or have better support for mixing said regular modes, 
something we already do.)

The new narrowing feature with labels kind of resembles that, except if 
I'm thinking about "intents", I could probably enumerate "visual" (for 
interactive narrowing), "hard" (for mixed-mode stuff or Info), or... 
"movement" (simply performed to constrain movement, but without goal to 
alter the parsing context -- such as narrowing before calling 
forward-sexp, in blink-matching-paren's example). These could be 
documented and assigned relative depth, sorting them like "hard > 
movement > visual", where code willing to undo "movement" would 
naturally undo "visual" as well, and code will to undo "hard" narrowing 
would undo them all.

How treesit-forward-sexp would handle "movement" narrowing is something 
up for discussion, though: maybe just ignore it (given that there is no 
performance tax), or maybe it would still perform a position check at 
the end (after finding the matching paren), to see if it ends up beyond 
the accessible region.