bug#57669: 29.0.50; C-n, C-p off under long lines

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Cc: 57669@debbugs.gnu.org
> Date: Fri, 09 Sep 2022 15:55:07 -0400
> 
> > showing your code that deals with very long lines and explaining how
> > it solves that
> 
> I can't believe I'm dignifying this.  All line references from commit
> beb489e.
> 
> xdisp.c[9319,9369]: Calculates dy algebraically without char-by-char.

This uses state variables from 'struct it' which are _local_ and
_momentary_, i.e. they can change as result of processing any buffer
position, and reflect only the latest change, forgetting what was
before.  Specifically, it->method is such a state variable.  So making
global conclusions, such as that "all characters will have the same
metrics on display", based on it->method is fundamentally wrong, and
can only be true in very simple situations.

The buffer->text->monospace flag is likewise implemented based on
simplistic assumptions: that only text properties could violate the
"monospace-ness" attribute.  Here's why this is simplistic:

  . faces can be applied by overlays as well
  . faces can be applied directly by C code
  . faces can be applied by using glyphs from display-tables
  . even if we only have the default face, fonts used for various
    non-ASCII characters can have different metrics from the default
    face's font, and in some cases they can even be variable-pitch
    fonts
  . portions of display can be made invisible by selective-display,
    not just by 'invisible' properties

(I'm not claiming that the above is the exhaustive list of all the
reasons that the assumptions in behaved_p could be violated; there
could be more of them.)

> xdisp.c[9537,9577]: Calculates CAPPED to limit char-by-char scan for
> moving backwards (Blandy and Moellmann wrote xdisp like a singly linked
> list -- fast and precise forwards, much less so going backwards).

I cannot see how the above description is related to the actual code
in that part of xdisp.c: there's no char-by-char movement back in the
corresponding parts of our code; instead, we move back by lines, then
move forward by characters -- and that's what the code there does as
well.  So I think this part of the code is equivalent to what we do,
in back_to_previous_visible_line_start, and should have the same
performance.  But maybe I'm missing something -- was this change
profiled, and if so, what were the results?  And if this code is
significantly faster, which of its changes is responsible for the
speedups?  And in which kind of files under what major modes were
these speedups measured?

In any case, the replacement code uses behaved_p, whose problematic
assumptions I already explained above.

To comment on the above description: in general, moving the iterator
backwards needs to consider all the changes in the state variables of
'struct it' that eventually affect the metrics and the layout
calculations.  It is _not_ the same as moving back by characters, and
the reason is that the state changes of 'struct it' are defined (in
set_iterator_to_next, get_next_display_element, and their subroutines)
only for moving forward, not for moving back.  That is why all
move_it_* functions cannot move back, except by going to a preceding
newline and then coming forward.