bug#51766: 29.0.50; Return value of buffer-chars-modified-tick changes when buffer text is not yet changed before inserting a character for non-latin input methods

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> That's exactly what happens: quail.el deletes the inserted character
> and then reinserts it (for reasons unrelated to this issue).  So the
> count of the changes is not equal to the number of characters actually
> inserted.  I see no problem here, since the documentation never
> promises that the difference between the values returned by successive
> calls to buffer-chars-modified-tick will be exactly equal to the
> number of inserted or deleted characters.

I agree that Emacs does not break any promises here. Though it is
unfortunate. Feel free to close this bug report.

> So if Org relies on such an equality, it's a bug in Org (but I didn't
> look at the relevant Org code, and don't have a clear idea of how
> exactly it uses the above function for whatever it is caching).

Let me explain a little (hoping that you might have some idea about
alternative solutions without using buffer-chars-modified-tick).

Org has a caching mechanism (org-element-cache) that keeps parsed buffer
representation in memory and updates it on the fly as the buffer
changes. To make the mechanism work, Org must keep track of all the
changes in buffer and update the affected Org elements in memory.
Naturally, this is done using before/after-change-functions.

However, some third-party code carelessly uses
inhibit-modification-hooks and some edits may be missed by element
cache. If we just ignore the possibility of such edits, cache can be
broken badly. So, there is currently a control code that detects if
buffer has been changed outside the Org's change functions. The control
code uses buffer-chars-modified-tick.

The behaviour of quail.el makes the control code useless -
buffer-chars-modified-tick can no longer be reliably used to detect
unfavourable "stealthy" changes. AFAIK, the only alternative way to
detect the changes is buffer-hash/secure-hash. But calculating hash is
very too slow when I try to put it into before/after-change-functions. I
do not know any fast (as fast as buffer-chars-modified-tick) way to
detect buffer changes.

> quail.el inhibit buffer modifications in places, since otherwise you'd
> have too many of them.  It wants to pretend that just one character
> was inserted.

I understand the idea behind suppressing the modification hooks by
quail. Though it would be helpful if before-change-functions were called
before inserting+deleting a character by quail is done.

Best,
Ihor