bug#36923: Combining Diacritical Marks are not Latin only

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Sun, 04 Aug 2019 23:40:38 +0300
> 
> The generated file lisp/international/charscript.el
> assigns the block “Combining Diacritical Marks” to the ‘latin’ script
> on the assumption that these characters are used only in Latin.
> 
> But in fact according to e.g. https://en.wikipedia.org/wiki/Acute_accent
> the acute accent marks the stressed vowel of a word in several languages
> with alphabets based on the Latin, Cyrillic, and Greek scripts.
> In particular https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode
> mentions how characters from other blocks are used in Cyrillic script.
> Moreover, the Combining Diacritical Marks block also
> contains several characters from the Greek script:
> COMBINING GREEK PERISPOMENI, COMBINING GREEK KORONIS
> COMBINING GREEK DIALYTIKA TONOS, COMBINING GREEK YPOGEGRAMMENI
> 
> I noticed this problem recently while helping to develop char-fold where
> GREEK SMALL LETTER IOTA combined with COMBINING GREEK DIALYTIKA TONOS was
> alarmingly highlighted as “mixed scripts” by markchars-mode from GNU ELPA.
> 
> Of course, it's possible to add exceptions for characters in this block
> in markchars-mode.  But before doing this, I'm asking a confirmation
> whether Unicode data should be fixed in ‘char-script-table’, so e.g.
> 
>   (aref char-script-table ?\N{COMBINING ACUTE ACCENT})
> 
> could return
> 
>   (latin greek cyrillic)
> 
> instead of the current
> 
>   latin

char-script-table is documented to yield a single symbol, so returning
a list would be an incompatible change, which we should avoid.

More generally, I think what you describe is a clear conceptual bug in
markchars-mode: it should only pay attention to the script of the base
characters, not to the script of combining accents.  The latter is
mostly irrelevant, certainly so for the purpose of detecting
confusables.

So I think this should be fixed in markchars-mode, and the fact that
we somewhat arbitrarily assign those diacritics to the latin script is
not a serious problem, if at all.