Re: Tree-sitter api

[Tuấn-Anh Nguyễn]

On Sat, Sep 4, 2021 at 10:00 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > 2. Who do we expect to build the grammar binaries?
>
> The ones that TS already provides?  They are already built, no?  Or
> what do you mean by "build the grammar binaries", what kind of
> binaries are those?  Forgive me my ignorance.

There are 2 components: TS the library (`libtree-sitter`) provides the generic
parts, not the grammars. The grammars come from various repositories, in source
form. (Some of them are owned by the tree-sitter project, some are not.) Each of
those consists of a generated `parser.c` and an optional `scanner.{c,cc}`. They
provide a function `TSLanguage (*tree_sitter_c) ()`, which specifies details on
how to parse a specific language (e.g. the parse table). They are usually
compiled into dynamically-loadable shared libraries (by a `tree-sitter` CLI
program), and distributed separately from `libtree-sitter`.

Tree-sitter has its own ABI versioning for these 2 components. It's easier to
ensure ABI compatibility if they are both built by the same system. That's the
case for GitHub's internal uses of tree-sitter. That's also the case with
`tree-sitter` and `tree-sitter-langs` packages on MELPA. That's not the case
with NeoVim's tree-sitter integration, and in a source of constant headache
AFAICT.

If we leave `libtree-sitter` to the distro, then it also makes sense for the
distro to provide the `tree-sitter` CLI program, and/or the grammar
binaries/sources.

--
Tuấn-Anh Nguyễn
Software Engineer