bug#50865: 28.0.50; Emoji with emoji modifier in Linux console garbles emacs display

[Aura Kelloniemi]

Hello,

I'm running emacs in the Linux virtual console. Showing text that contains
Unicode emoji characters with modifiers causes wrong output. The problem may
be related to the fact that Linux basically does not understand Unicode
composition, multi-column charactes or anything fancy. Anyhow, here are the
instructions to reproduce this:

Run emacs -Q

In the scratch buffer on an empty line:
Type C-x 8 RET PERSON WITH FOLDED HANDS RET
🙏 appears followed by the cursor. This is what I expect. (In the TTY screen
only a black diamond is shown, but I can verify from /de/vcsu that the
character really is an emoji.)

Then type C-x 8 RET EMOJI MODIFIER FITZPATRICK TYPE-3 RET
Now the emoji modifier appears (again as a diamond), but it is followed by a
space character.

So now the line looks like:
<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE>
_cursor_

I think that the space there should not be added. My guess is that the space
is actually not written to the terminal, but that emacs misplaces the cursor.

Now if a press backspace to delete the last typed character (the emoji
modifier), only the space disappears. If I then press backspace again, both the
emoji and the modifier disapper at the same time.

Interestingly, if I repeat the process of adding the above mentioned emoji and
the modifier characters on a line two times (starting again from an empty
line), the line looks like this:

<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> <PERSON
WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> _cursor_

If I now run M-x redraw-display RET, the line looks like this:

<PERSON WITH FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <PERSON WITH
FOLDED HANDS> <EMOJI MODIFIER FITZPATRICK TYPE-3> <SPACE> <SPACE> <SPACE>
_cursor_

So for some reason there are now three spaces before the cursor.

I tried this on Debian version of Emacs 26.1, and the results were similar,
but not exactly identical.

Why do I care? If I use a Linux console, and it cannot display emoji, why does
this matter? Because if there is any other text on the same line, it often
gets very garbled, especially if Emacs decides to only update the line
partially.

I cannot stop other people from using emojis nowadays, and that's why I'd like
Emacs to tolerate them.

I will gladly provide more detail. I'm also interested in any (dirty) hacks
that could be used to work around this issue, as it disturbs my emacs usage
all the time (I use Telegram from within Emacs).

If I run emacs within GNU screen (that itself runs in a Linux VT), this
problem does not seem to appear. There are other issues with some Unicode
characters in screen, but I haven't yet found a clear way to reproduce these
issues.

Below is the data produced by M-x report-emacs-bug RET






In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo 
version 1.17.4)
 of 2021-09-26 built on solaria
Repository revision: 43ae8c828d853382bbc2a27b9e14b9fff6ba18b6
Repository branch: makepkg
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-native-compilation --with-x-toolkit=gtk3
 --with-xft --with-wide-int --with-modules --with-gameuser=:games
 --with-sound=alsa --with-cairo --with-harfbuzz
 --enable-link-time-optimization 'CFLAGS=-march=native -mtune=native -O2 -pipe
 -fno-plt -fuse-ld=gold -flto -fuse-ld=gold -flto'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
 CPPFLAGS=-D_FORTIFY_SOURCE=2'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM
XPM GTK3 ZLIB

Important settings:
  value of $LANG: fi_FI.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  gpm-mouse-mode: t
  leaf-key-override-global-mode: t
  shell-dirtrack-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  icomplete-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: linux
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Load-path shadows:
/home/aura/.config/emacs/elpa/magit-20210925.1143/magit-section-pkg hides 
/home/aura/.config/emacs/elpa/magit-section-20210829.1849/magit-section-pkg
/home/aura/.config/emacs/elpa/transient-20210919.1006/transient hides 
/usr/share/emacs/28.0.50/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail cursor-sensor t-mouse term/linux
desktop+ haskell-mode haskell-cabal haskell-utils haskell-font-lock
haskell-indentation haskell-string haskell-sort-imports haskell-lexeme
haskell-align-imports haskell-complete-module haskell-ghc-support flymake-proc
flymake warnings dabbrev haskell-customize python tramp-sh autorevert
filenotify conf-mode cargo cargo-process markdown-mode color racer f s dash
company-oddmuse company-keywords company-etags etags fileloop xref project
company-gtags company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-cmake company-semantic company-template company-bbdb
vc-git vc-dispatcher rust-utils rust-mode rx rust-rustfmt rust-playpen
rust-compile compile rust-cargo yaml-mode org-element avl-tree generator
ol-eww eww xdg url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-search eieio-opt cl-extra help-mode speedbar ezimage dframe
gnus-art mm-uu mml2015 gnus-sum shr kinsoku svg dom gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader ol-docview doc-view
jka-compr image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m org ob ob-tangle
ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table ol org-keys org-compat advice org-macs
org-loaddefs find-func recentf tree-widget notmuch notmuch-tree notmuch-jump
notmuch-hello notmuch-show notmuch-print notmuch-crypto notmuch-mua
notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address
notmuch-company notmuch-parser notmuch-wash diff-mode easy-mmode coolj
notmuch-query goto-addr thingatpt icalendar diary-lib diary-loaddefs cal-menu
calendar cal-loaddefs notmuch-tag crm notmuch-lib notmuch-version
notmuch-compat hl-line message rmc puny dired dired-loaddefs rfc822 mml
mailabbrev gmm-utils mailheader mm-view mml-smime mml-sec epa derived epg
rfc6068 epg-config gnus-util rmail rmail-loaddefs mail-utils
text-property-search smime dig mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr server leaf-keywords
leaf finder-inf package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse url-vars cus-edit pp wid-edit tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat shell pcomplete comint ansi-color ring
parse-time iso8601 time-date ls-lisp format-spec auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json map savehist
minibuf-eldef keypad ido seq byte-opt gv bytecomp byte-compile cconv icomplete
company edmacro kmacro pcase subr-x desktop frameset cl-loaddefs cl-lib
cus-load info iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs
faces cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 541107 17966)
 (symbols 48 48459 2)
 (strings 32 220875 3286)
 (string-bytes 1 6213483)
 (vectors 16 57360)
 (vector-slots 8 1387450 51457)
 (floats 8 484 304)
 (intervals 56 910 2)
 (buffers 992 39))

-- 
Aura