bug#20847: [display engine] 25.0.50; company-mode popup makes point jump to an entirely different location

[Eli Zaretskii]

> Cc: 20847@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 21 Jun 2015 16:56:56 +0300
> 
> On 06/20/2015 02:51 PM, Eli Zaretskii wrote:
> 
> > In a nutshell, when a screen line ends in a newline that comes from an
> > overlay string, we don't want to display the cursor on that line.  The
> > reasons are heuristic, but they give good results, and we used this
> > heuristic for a very long time, so changing it now is out of question.
> 
> Do you have a scenario in mind that performs better under the current 
> behavior?

Any scenario where a screen line ends in a newline that comes from an
overlay string.  Try several such scenarios, and then tell me whether
the place we display the cursor looks better than the alternative.

> I can understand the scenario until now, but why move point?
> 
> Displaying cursor in a different place is relatively fine, but moving 
> the point is destructive.

Emacs cannot move cursor except by moving point, I'm sure you know
that.  The only exception is when we show the cursor on a display or
overlay string, guided by the 'cursor' property.  There are no other
exceptions.

> > The font-lock part of this riddle is that when font-lock-mode is
> > active in the buffer, making any changes to buffer text cause JIT Lock
> > to spring to action, which doesn't really do anything, but disables a
> > certain redisplay optimization, which bypasses the above test.
> 
> Sounds messy.

You can say that again.

> > My suggestion would be to use the 'cursor' property on the overlay
> > string in some place where it could be picked up by the display engine
> > (i.e. not on a newline), to countermand this problem.
> 
> See the bottom of `company--replacement-string'. If `cursor' is applied 
> unconditionally, and if I change the arguments 0 and 1 to 1 and 2, on 
> step 6 the cursor is displayed at the beginning of the next line (so we 
> know the change has effect), but the second problem (after step 9) is 
> still present.

AFAICT, this will put the 'cursor' property on a character that is
after the leading newline of the overlay string, yes?  If so, that's
not going to work: you need th 'cursor' property on some glyph that is
displayed on the same line where the newline is.  That is, you need to
make at least one character of "hel" part of the overlay string, and
put the 'cursor' property on it, making its value large enough to
"cover" the position of the newline.

> > E.g., perhaps
> > begin the overlay string a few characters earlier, so that it replaces
> > part of buffer text in "hel", and have the 'cursor' property on that
> > part of the string.
> 
> That's doable, even if I don't like the extra complexity. Are you sure 
> about this?

I didn't have time to actually try that, I just looked at the code.
So it might not work as things are, but if so, I'm quite sure I can
fix that.  As long as some glyph that came from the overlay string is
visible on the same line, and the corresponding string character has a
'cursor' property on it, the display engine has enough information to
decide that the cursor can be displayed there (on the fringe).

> Do you also have explanations for the following?
> 
> - The bug only manifests after the step 9 (backspacing), whereas the 
> whole explanation seems to apply to the step 6 as well. Yet, point stays 
> in place there.

Like I said, I didn't investigate that.  I think some redisplay
optimization is responsible.  If it's important to have the same
(mis)behavior in both cases, I can look into that.

> - With bidi-display-reordering set to nil, there's no bug.

Because the unidirectional display could assume that buffer positions
increase monotonically with the screen's vertical coordinate, and so
once it saw a single screen line whose first and last characters have
buffer positions on both sides of point, it could decide to put the
cursor on that line right there and then; it didn't need to test other
conditions or consider following screen lines.  So this situation
worked "by sheer luck".  The bidirectional display doesn't have that
luxury, so it needs additional support information, and that
information is simply absent in this case.