Re: Recent updates to tree-sitter branch

[Yuan Fu]

> On Sep 26, 2022, at 2:43 AM, Ihor Radchenko <yantar92@gmail.com> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> treesit-font-lock-rules rules take a form of
>>> (MATCHER FACENAME) or (MATCHER FUNCTION)
>>> 
>>> where MATCHER can only be a query.
>>> 
>>> Is there any reason why MATCHER in treesit-font-lock-rules cannot be a
>>> function with access to the fontified node?
>> 
>> Hmm, I’m not sure what do you mean. The whole thing passed to 
>> treesit-font-lock-rules is a single query, and we can’t really change the 
>> query syntax, that’s defined by tree-sitter. Basically in a query you have 
>> patterns paired with capture names, if the pattern matches to a node, that 
>> node is returned with corresponding capture name tagged on it. For 
>> font-lock, we just use face names as capture names, and when a query returns 
>> captured nodes, fontify the node with its capture name, aka a face (or a 
>> function).
> 
> What I am asking is an extra dynamic condition in addition to the query.
> For example:
> 1. Only apply FACENAME for nodes matching QUERY, but only when Elisp
>   variable is non-nil
> 
> 2. Only apply FACENAME for nodes matching QUERY, which are in the second
>   half of the buffer
> 
> 3. Only apply FACENAME for notes matching QUERY, which also have a field
>   matching a dynamically assigned regexp.
> 
> Essentially any condition that is not covered by the QUERY, but can be
> checked in Elisp given that node object is passed to the test function.

These can be achieved by using a function, no? You do need to declare global 
functions for them, but it shouldn’t be a problem. Besides, as I said, the 
query syntax is not something we can change. The freedom we have is how do we 
use the capture names. We can’t extend the query with arbitrary lisp.

> 
>>> Further, can OVERRIDE FLAG of the MATCH-HIGHLIGHT as in
>>> font-lock-keywords be supported?
>>> 
>>>  "If OVERRIDE is t, existing fontification can be overwritten. If
>>>   keep, only parts not already fontified are highlighted. If prepend or
>>>   append, existing fontification is merged with the new, in which the new
>>>   or existing fontification, respectively, takes precedence.”
>> 
>> I can do that, but would it be really useful? Unlike regex font-lock which 
>> is used for so many different things, tree-sitter font-lock is, IMO, only 
>> used to apply a base layer of language-specific highlight. How would one use 
>> the override feature in this scenario?
> 
> For example, consider a function definition with docstring field.
> Imagine that you want the function definition to have gray background,
> but the docstring to have yellow background. OVERRIDE t is how this is
> usually implemented in font-lock-keywords.

The pattern that comes after will override patterns that come before. By the 
nature of parse trees, for any node A and another smaller node B, B is either 
completely contained in A or completely outside A. So I think the override 
relationship is enough.

Yuan