Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 11/11/2021 13:50, Vladimir Lomov wrote:
>> ** Ihor Radchenko:
>> 
>>> Vladimir Lomov writes:
>>>> Warning (emacs): org-element--cache: Unregistered buffer modifications 
>>>> detected. Resetting.
>> 
>>> Are you able to reproduce with emacs -Q?
>
> I can confirm it starting with a simple file
>
> ---- >8 ----
> | 1 |
> ---- 8< ----
>
> LANG=en_US.UTF-8 emacs -Q -L ~/src/org-mode/lisp/ cyrtable.org
>
> C-\ russian-computer RET to switch input method
> TAB to create a new cell
> Any letter

Confirmed

I can also reproduce with russian-computer and at least arabic. Seems to
be an issue with non-latin input methods.

The warning is triggered because return value of
buffer-chars-modified-tick with non-latin input method changes _before_
text is inserted. If I add debug-on-entry for self-insert-command or
org-self-insert-command, buffer-chars-modified-tick changes twice: (1)
some time after pressing the keyboard key but before entering
self-insert function (symbol is not inserted into buffer); (2) after
actual insertion. The problem happens in non-Org buffers as well and
with emacs -Q.

org-element-cache relies on the return value of
buffer-chars-modified-tick to control if all the changes in buffer are
reflected in the cache. The docstring says:

>> By comparing the values returned by two individual calls of
>> buffer-chars-modified-tick, you can tell whether a character change
>> occurred in that buffer in between these calls

So, what we observe looks like a Emacs bug.

On Org side, this bug is very bad news. We cannot wait for the Emacs
fixing the bug - older Emacs versions will still be affected.
Alternative ways to control buffer modifications are buffer-hash and
secure-hash. In the past, I had some random failures when buffer-hash
did not reliably reflect buffer updates. The only alternative is
secure-hash, but I am not sure about it's performance. buffer-hash
docstring says that "It should be somewhat more efficient on larger
buffers than secure-hash is, and should not allocate more memory.". So,
I need to test the actual performance on large buffers before switching
from buffer-chars-modified-tick to secure-hash.

Also, I am somehow unable to reproduce the problem in my private Org
branch. Maybe there is some alternative fix without getting rid of
buffer-chars-modified-tick.

Vladimir, if the issue is affecting your workflows, you can disable
org-element-cache until we fix the bug. Just set org-element-use-cache
to nil in your config before loading Org.

Best,
Ihor