Re: font-lock-extend-region

[Alan Mackenzie]

Hi, Stefan!

On Mon, 20 Mar 2006, Stefan Monnier wrote:

>[ I hope some of this exchange can get into some documentation somewhere.
>  I find it very useful to have to explain it in such detail. ]

Agreed on both points!  :-)

>> I think there are two distinct issues here that we're confusing, and this
>> is why we've found it so hard to agree:

>>     (i) calculating the region which needs refontifying.
>>     (ii) finding a safe place to start fontifying a single chunk.

>Right.  The first is generally handled by jit-lock-context-* (including the
>jit-lock-defer-multiline property).  The second is generally handled by the
>font-lock-multiline property and by rounding up to a whole number of lines.

>> font-lock-extend-region-function is intended to do (i).  The
>> functionality you're suggesting for f-l-default-fontify-region is for
>> doing (ii).

>In other words your use of font-lock-extend-region-function is specifically
>to deal with issues that jit-lock-context-* tries to handle as well.

Er, maybe.  ;-)

>> I think the essence of the font-lock-multiline property is that it
>> marks a chunk of text to be fontified atomically.  Please confirm this
>> impression or correct it for me.

>Correct.

OK, time for me to document this in modes.texi, I think.

>> Here's why I think the font-lock-multiline way is wrong.  Taking my AWK
>> example again:

>> 1. "string \
>> 2. over \
>> 3. several \       <=========
>> 4. #lines."

>> Suppose the user replaces the backslash on L3 with 20k of code from the
>> kill ring with M-y.  The region to fontify now extends from L1 to EOL4
>> (actually, it's now L1073).  The display engine is going to request
>> fontification from L1034.  If I mark this entire region with
>> font-lock-multiline, these 1073 lines will be (unnecessarily) fontified
>> atomically, defeating the aims of jit-lock in this case.

>The region is not automatically marked with font-lock-multiline, so you
>can't really fault font-lock-multiline for it: it's your code that marks it
>that is at fault.
>[ Now don't get me wrong: the font-lock-multiline property is not perfect. ]

Hang on a moment!  The CC Mode code _doesn't_ currently mark text with
font-lock-multiline.  I understood your suggestion was that it should do
this.

>> What I think we need is a function called from f-l-default-f-region which
>> will get a safe starting position at or before L1034.

>Agreed.  And I suggested we name it font-lock-extend-region-function.

font-lock-extend-region-function is already in use for purpose (i),
determining the region to be fontified.  I'm saying we also need a
distinct function for purpose (ii), determining a safe place to start
fontifying a chunk which could easily be in the middle of the region
returned by (i).

>You seem to be saying that you'd also want such a thing in
>after-change-functions.

Yes, that's exactly what I'm saying.

>My belief is that you don't need it for the following reason: if you need to
>refontify more than the 20K of code you just yanked, it can only be because
>of elements at the beginning/end that need to be refontified atomically, so
>you can just either place a font-lock-multiline property on them or extend
>the region from f-l-default-fontify-region.

   1. "string \
   2. over \
   3. several    <========= backslash/NL here replaced by ....
<....20k of text from kill ring.>
1073. #lines."

>But maybe this is only true in theory, and reasons of performance (or
>presence/absence of various info in different contexts) make it that you
>do need an "extend-region" in font-lock-after-change-function.  Is that
>what you are saying?

Yes.

After a buffer change, the region can't be extended within
f-l-default-fontify-region, since that function doesn't have access to to
BEG, END and OLD-LEN of after-change-functions, nor to OLD-BEG and
OLD-END of before-change-functions.  This analysis _must_ be done within
an after-change function.

On the other hand, determining the boundaries of a chunk of text that
needs fontifying atomically _must_ be done in f-l-default-f-region.  I
can't see any advantages in doing it in the after-change function too.

It is also conceivable that f-l-extend-region-f returns a start-position
well before the change, yet the interval from there to the change
position could be fontified in several distinct chunks rather than
atomically.  I admit I can't think of an realistic example right now,
though.

>Or is there some other reason to extend the region from
>after-change-functions, other than atomic elements at the boundaries?

The region sometimes has to be extended to the end of what was an atomic
element before the change, but is no longer so.

>> What I think we should do is to put a hook into f-l-default-f-region
>> to calculate a safe starting position (and probably also a safe
>> stopping position).

>Yes, we agree on that.

:-)

>> Incidentally, referring to my diagram above, the region gets extended
>> to whole lines more than once.  For demand fontification, it is done
>> first in jit-lock-fontify-now then in
>> font-lock-default-fontify-region.  For after-change fontification, it
>> is done yet a third time in jit-lock-after-change.

>Yes, it's a bit messy, partly for historical reasons.

>> How about doing this only in f-l-default-f-r?  This would make it easier
>> for a mode maintainer to switch off this action, since he would then just
>> have to put a modified function into the hook
>> font-lock-fontify-region-function.

>I believe the one in jit-lock-fontify-now could be removed (but this
>function is also sadly called from external packages, so there may be some
>minor compatibility issues).

I think any text region handled by jit-l-f-now cannot avoid being
processed by f-l-fontify-region.  This means f-l-default-f-region, unless
the hook f-l-f-region-function has been changed.  Somebody who does this
surely knows what she is doing.

>But the one in jit-lock-after-change is needed because of what the
>comment there says.  Basically here is the scenario:

>  start with C code like

>    void foo 

>  and add an open parenthesis at the end.  The modified chunk is just the
>  open paren, so if you don't reset the `fontified' property on the whole
>  line, the redisplay engine will not redisplay `foo' and even if jit-lock
>  changes the `face' property on `foo' it does it after the display engine
>  decided what `foo' would look like.  So if jit-lock-after-change doesn't
>  round up to whole lines, `foo' in the above scenario would only be
>  refontified at the next screen refresh :-(

I don't see this, yet.  If 'fontified is cleared only on the new open
paren at position 1055, the display engine calls successively
  (jit-lock-function 1055),
  (jit-lock-fontify-now 1055 1555),
    and (assuming j-l-f-now DOESN'T extend to whole lines)
  (run-hook jit-lock-functions 1055 1555),
  (font-lock-fontify-region 1055 1555),
  (funcall font-lock-fontify-region-function 1055 1555),
  (font-lock-default-fontify-region 1055 1555).
Here, in f-l-default-f-r, 1055 is set back to 1044 (BOL), and this whole
line, 1044 - 1056, gets its face properties rearranged.  These then get
redisplayed.  What am I missing here?

>I'd like to be able to solve this problem elsewhere than in
>jit-lock-after-change (e.g. some way for jit-lock to say "hey, font-lock
>modified this text before BEGIN, please make sure you redisplay it
>immediately"), but even if I knew how to do that, I'd probably not use it
>for whole-line-round-up because it would simply cause (almost) all redisplay
>to be done twice.


>        Stefan

-- 
Alan.