Re: Why tree-sitter instead of Semantic? (was Re: CC Mode with font-lock-maximum-decoration 2)

[Lynn Winebarger]

CC'ing Yuan Fu on Po Lu's recommendation earlier in the thread.

On Wed, Aug 10, 2022 at 7:31 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Wed, 10 Aug 2022 06:05:56 -0400
> > Cc: Po Lu <luangruo@yahoo.com>, Alan Mackenzie <acm@muc.de>, emacs-devel 
> > <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>
> >
> >  It is simply too slow to be a modern solution for these features.
> >
> > Can you (or anyone on the list) provide a more detailed analysis?  Is the 
> > slowness inherent in the algorithm
> > design, the implementation method (eg table driven parsing designed before 
> > the availability of the native
> > compiler), the basic synchronous nature of ELisp, the impact on garbage 
> > collection, etc?
>
> I don't have this information.  Maybe someone else does.  But in
> general, it is a very small wonder that a parser written in optimized
> C is much faster than anything written in Emacs Lisp, given that Lisp
> is an interpreted language that has no special support for writing
> parsers.

That can be cured over time, now that the bulk of the core of emacs
uses lexical scoping.  With proper tail recursion, ELisp should be
able to produce lexers and parsers roughly as efficient as C code, if
not more efficient (depending on if you allow use of "computed goto"
in the C code for the lexers and parsers).  That does require changes
to the byte code VM, but it's doable.

>
> > If the analyzer were run in a second emacs process using mmaped files to 
> > share buffers being analyzed,
> > then communicating the results either via LSP or some other channel, would 
> > that make it usable?
>
> I doubt that.  In particular, LSP-style communications are a cause of
> slower operation, not faster operation.  Various LSP-based packages
> tolerate that because the server can do stuff clients cannot easily do
> without investing a lot of language-specific efforts and expertise.

That's more along the lines of what I expected (and Tassilo's
response) - the overhead of creating and maintaining these language
analyses in Emacs.  But for a DSL, I'd personally prefer to be able to
put something together "quickly" in emacs to get a working IDE.

> > I've definitely noticed more pausing with Semantic turned on, but it's not 
> > unusable so far (but I'm also not
> > looking at any C++ source, just ELisp and C, maybe some shell scripts, info 
> > files and Markdown).
>
> I didn't say Semantic is unusable.  It certainly is usable.

The pausing was in fact due to a different library - tabbar-ruler -
that had for some reason made frequently refreshing the tab list call
eval on a thunk instead of just calling the thunk, generating huge
amounts of garbage from "eval".  Since I corrected that, I haven't
noticed much pausing.  A better fix would make it stop refreshing tabs
that aren't even displayed when Semantic takes over the header line,
but that would require more effort.

Lynn