Re: Reliable after-change-functions (via: Using incremental parsing in Emacs)

[Eli Zaretskii]

> From: Stefan Monnier <address@hidden>
> Cc: Yuan Fu <address@hidden>,  address@hidden,  address@hidden
> Date: Mon, 30 Mar 2020 23:10:57 -0400
> 
> > IOW, our goal is not to build the syntax tree, it's to give
> > tree-sitter enough information to allow us to fontify the part that's
> > about to be displayed.  We need to have tree-sitter play by Emacs
> > rules, not teach Emacs to play by tree-sitter rules.
> 
> IIUC, tree-sitter starts by parsing the whole buffer anyway, and then
> keeps the parse tree up-to-date in response to buffer changes.

Why does it need the entire buffer up front? that sounds like a
potential performance killer.  Fontifying a small part of a buffer
doesn't need its entire text.

In any case, I hope that passing the buffer to tree-sitter doesn't
involve marshalling the entire buffer text via a function call as a
huge string, or some such.  We should instead request that tree-sitter
exposes an API through which we could give it direct access to buffer
text as 2 parts, before and after the gap, like we do with regex
code.  Otherwise this will be a bottleneck in the long run, not unlike
the problem we have with LSP.

> Its algorithm is tuned so that the time needed to update the tree is
> more or less proportional to the size of the change.
> 
> So jit-lock/font-lock doesn't need to pass any part of the buffer to
> tree-sitter: tree-sitter already has the buffer's content and we can
> assume its already parsed.  What emacs-tree-sitter's proposed
> tree-sitter-highlight does is provide a function which takes
> a START..END, then finds which part of the existing parse tree cover
> that region and "reads the tree" to fontify the corresponding text.

I still don't see why it would need the entire buffer for this class
of applications.  Did anyone try the alternatives, in particular on
very large buffers?