bug#53155: Use overlays in hi-lock

[Juri Linkov]

When diff-refine overlays take precedence over hi-lock text properties,
then hi-lock can't be used to review diffs.  Hi-lock provides two modes:
font-lock highlighting and highlighting with overlays.  Using overlays
solves the problem.  But how to decide when overlays should be used?
Detecting whether the buffer is read-only, so no more text is expected
to be added to the buffer?  Then add overlays instead of using font-lock?
But this heuristics doesn't work in read-only *vc-change-log* buffers
where expanding commits inserts new text that should be highlighted
with font-lock.

Thus the following patch adds a new variable hi-lock-use-overlays
that can be used like:

#+begin_src emacs-lisp
(add-hook 'diff-mode-hook
            (lambda ()
              (setq-local hi-lock-use-overlays t)))
#+end_src

diff --git a/lisp/hi-lock.el b/lisp/hi-lock.el
index fbd698e234..99e908ad07 100644
--- a/lisp/hi-lock.el
+++ b/lisp/hi-lock.el
@@ -738,6 +738,17 @@ hi-lock-read-face-name
       (add-to-list 'hi-lock-face-defaults face t))
     (intern face)))
 
+(defvar hi-lock-use-overlays nil
+  "Whether to always use overlays instead of font-lock rules.
+When font-lock-mode is enabled and the buffer specifies font-lock rules,
+highlighting is performed by adding new font-lock rules to the existing ones,
+so when new matching strings are added, they are highlighted by font-lock.
+Otherwise, overlays are used, but new highlighting overlays are not added
+when new matching strings are inserted to the buffer.
+However, sometimes overlays are still preferable even in buffers
+where font-lock is enabled, because hi-lock overlays take precedence
+over other overlays in the same buffer, such as diff-refine overlays, etc.")
+
 (defun hi-lock-set-pattern (regexp face &optional subexp lighter case-fold 
spaces-regexp)
   "Highlight SUBEXP of REGEXP with face FACE.
 If omitted or nil, SUBEXP defaults to zero, i.e. the entire
@@ -759,7 +770,8 @@ hi-lock-set-pattern
         (add-to-list 'hi-lock--unused-faces (face-name face))
       (push pattern hi-lock-interactive-patterns)
       (push (cons (or lighter regexp) pattern) hi-lock-interactive-lighters)
-      (if (and font-lock-mode (font-lock-specified-p major-mode))
+      (if (and font-lock-mode (font-lock-specified-p major-mode)
+               (not hi-lock-use-overlays))
          (progn
            (font-lock-add-keywords nil (list pattern) t)
            (font-lock-flush))
@@ -781,6 +793,7 @@ hi-lock-set-pattern
                                            (match-end subexp))))
                 (overlay-put overlay 'hi-lock-overlay t)
                 (overlay-put overlay 'hi-lock-overlay-regexp (or lighter 
regexp))
+                (overlay-put overlay 'priority 1)
                 (overlay-put overlay 'face face))
               (goto-char (match-end 0)))
             (when no-matches