Re: Tree-sitter integration on feature/tree-sitter (severe performance issues together with linum-mode)

[Yuan Fu]

> On Sep 5, 2022, at 7:53 PM, Clément Pit-Claudel <cpitclaudel@gmail.com> wrote:
> 
> On 8/20/22 07:14, Stefan Monnier wrote:
>> FWIW, I think specifying the highlighting rules with something akin to:
>>     (defvar <foo> '<rules>)
>> is a mistake.  It should go through some kind of macro, such as (maybe):
>>     (defvar <foo> (tree-sitter-rules <rules>))
>> which can thus do any preprocessing we may want, such as pre-compiling
>> queries.  It also helps evolve the syntax since we can more easily warn
>> about obsolete uses, etc...
>> I've had a "rewrite font-lock.el so the rules go through a macro" in my
>> todo list for ages.
> 
> We do things this way in Flycheck, but we've been bitten a few times by the 
> way this pattern interacts with `with-eval-after-load`, so I would be careful 
> about adopting this pattern in tree-sitter (unless we expect it to be 
> preloaded).
> 
> In fact, I think your suggestion back then was to *not* use a macro?  I can't 
> find the exact thread, but this is close enough: 
> https://lists.gnu.org/archive/html/emacs-devel/2018-02/msg00820.html :)
> 

I see Stefan’s follow-up in the new thread, but in terms of tree-sitter, I 
actually defined it as a function rather than a macro (because the habit of 
always preferring functions if a function will do). IIUC function doesn’t have 
the same problem as macros (when byte-compiling)? It is used like this:

(treesit-font-lock-rules
 :language 'c
 '((false) @font-lock-constant-face))

Definition:

(defun treesit-font-lock-rules (&rest args)
  "Return a value suitable for `treesit-font-lock-settings'.

Take a series of QUERIES in either string, s-expression or
compiled form.  Same as in `treesit-font-lock-settings', for each
query, captured nodes are highlighted with the capture name as
its face.

Before each QUERY there could be :KEYWORD VALUE pairs that
configure the query.  For example,

    (treesit-font-lock-rules
     :language javascript
     '((true) @font-lock-constant-face
       (false) @font-lock-constant-face
     :language html
     \"(script_element) @font-lock-builtin-face\")

For each QUERY, a :language keyword is required.  Currently the
only recognized keyword is :language.

\(fn :KEYWORD VALUE QUERY...)"
  (let (;; Tracks the current language that following queries will
        ;; apply to.
        (current-language nil)
        ;; The list this function returns.
        (result nil))
    (while args
      (let ((token (pop args)))
        (pcase token
          (:language
           (let ((lang (pop args)))
             (when (or (not (symbolp lang)) (null lang))
               (signal 'wrong-type-argument `(symbolp ,lang)))
             (setq current-language lang)))
          ((pred treesit-query-p)
           (when (null current-language)
             (signal 'treesit-error
                     `("Language unspecified, use :language keyword to specify 
a language for this query" ,token)))
           (if (treesit-compiled-query-p token)
               (push `(,current-language token) result)
             (push `(,current-language
                     ,(treesit-query-compile current-language token))
                   result))
           ;; Clears any configurations set for this query.
           (setq current-language nil))
          (_ (signal 'treesit-error
                     `("Unexpected value" token))))))
    (nreverse result)))

Yuan