font lock with functions (was: emacs rendering comparisson between emacs23 and emacs26.3)

[David Engster]

> On 29.03.2020 19:39, Stefan Monnier wrote:
>> I think it's just a reflection of the fact that author doesn't know
>> enough about jit-lock and font-lock.
>
> Indeed, that font-lock rule can be a function is not very well-known.

Indeed. Since I'm currently writing a major-mode which makes heavy use
of this feature, let me add my experience.

I found the font-lock documentation in the Emacs Lisp reference manual
to be pretty opaque, but I also wouldn't really know how to improve
it. What is mostly missing are practical examples, but I understand that
a reference manual is probably not the right place for this. I found
some helpful posts online, like this SO answer from you

https://stackoverflow.com/a/14675550

and this blog post

http://www.modernemacs.com/post/advanced-syntax/

Other smaller problems I'm having:

- It is very easy during testing to completely hang your Emacs session
  if one of the font-lock functions hangs. One usually needs to kill
  Emacs in this case.

- Debugging is difficult because you cannot instrument the functions for
  some reason. Luckily, I found 'font-lock-studio', which is incredibly
  useful. We should try to get this into ELPA and advertise it in the
  manual:

   https://github.com/Lindydancer/font-lock-studio

On a more general note: When you use functions for font-locking, I often
find myself in the position that I'm more or less writing a parser, and
I could readily tell font-lock how to fontify several parts of the
buffer with different faces. However, AFAICS, this is not possible with
current font-lock? It seems each rule can always only return one type of
face, or am I missing something? Of course, you can still make this
work, but it makes font-lock pretty slow, because you effectively need
to re-parse the changed portions of the buffer several times. I would
imagine that fontifying based on tree-sitter would run into the same
issue?

-David