Re: New package emacs-parser-generator

[Yuan Fu]

The tree-sitter integration doesn’t provide anything to define a grammar nor 
generating a parser. It merely uses pre-defined grammar (written by tree-sitter 
community) to parse buffers and produce an AST.

To write an tree-sitter grammar definition you need to write the grammar in 
JavaScript and pass it to tree-sitter’s parser generator. The generator spits 
out a grammar definition encoded in a C source file (in the form of a C 
struct). To use this grammar definition, we compile it to a library, load it at 
runtime, pass the struct to tree-sitter library, and tree-sitter can now parse 
according to this grammar definition.

The tree-sitter integration provides 1) Lisp wrappers for tree-sitter’s C API; 
2) some convenient functions built on the C API; 3) integration with font-lock 
and indentation. I didn’t add any “common API”, I simply used existing 
frameworks in Emacs: for font-lock I used font-lock-fontify-region-function, 
for indent I used indent-line-function. If in the future a common API for 
parses is desirable, tree-sitter can easily comply.

IOW, this is what tree-sitter integration currently does:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
tree-sitter-font-lock-fontify-region     tree-sitter-indent-function

This is what could happen if we want a common API:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
     common  API  --------------------------+
     /         \
    /           \
   /             \
tree-sitter    other parser

Yuan