bug#37530: 26.1; Tack characters translated incorrectly

[Axel Svensson]

On Fri, Sep 27, 2019, 07:27 Eli Zaretskii <eliz@gnu.org> wrote:
>
> Isn't the output of "C-h l" evidence that Emacs actually received the
> codepoints it displayed?  IOW, how do we know this is a problem in
> Emacs and not in the keyboard configuration and/or driver software?
>
> Thanks.

Good point. The following tests confirm that the mistranslation is in
Emacs, and specifically in its handling of X11 events, and not in the
setup:

- Install xdotool (On debian, `apt-get install xdotool`)
- In order to test an application's tack character interpretation:
  - In a terminal, run:
    sleep 3; xdotool \
      key --delay 500 uptack \
      key --delay 500 downtack \
      key --delay 500 lefttack \
      key --delay 500 righttack
  - Quickly switch focus to the application you want to test
  - Wait 5 seconds
  - Investigate the effect in the application. If the application is
    such that it attempts to display characters that are inputted via
    the keyboard, it should display the string "⊥⊤⊣⊢".

On my setup, the following are the results of using the test above for
emacs, compared to a few other applications:

- GNU Emacs 26.1 started with -Q:
  The string "⊤⊥⊢⊣" is displayed, which is wrong.
- Gnome Terminal 3.30.2:
  The string "⊥⊤⊣⊢" is displayed, as expected.
- GNU Emacs 26.1 started with -Q -nw under Gnome Terminal 3.30.2:
  The string "⊥⊤⊣⊢" is displayed, as expected.
- Chromium 73, with focus in the address bar:
  The string "⊥⊤⊣⊢" is displayed, as expected.
- Firefox 60.8.0esr 64-bit, with focus in the address bar:
  The string "⊥⊤⊣⊢" is displayed, as expected.
- xterm:
  No characters are displayed.
- xev (in package xutils, x11-utils, or similar):
  This application prints debugging information for X11 events. See
  the output below. It confirms that the setup is sending correct key
  events. However, XLookupString refuses to translate the keysym to a
  utf-8 string. I believe this might mean that it is up to the
  applications to find a way to translate the keysyms, and can explain
  why some applications do not display any characters. However, it does
  not explain why emacs displays the wrong characters.

===== Begin relevant part of xev output =====
MappingNotify event, serial 32, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 32, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 32, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 32, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyPress event, serial 36, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141855379, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbce, uptack), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 36, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyRelease event, serial 36, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141855632, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbce, uptack), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 44, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyPress event, serial 44, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141855885, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbc2, downtack), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 52, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyRelease event, serial 52, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141856137, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbc2, downtack), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 60, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyPress event, serial 60, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141856392, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbdc, lefttack), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 68, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyRelease event, serial 68, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141856643, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbdc, lefttack), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 76, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyPress event, serial 76, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141856898, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbfc, righttack), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 84, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

KeyRelease event, serial 92, synthetic NO, window 0x2c00001,
    root 0x388, subw 0x0, time 141857162, (513,324), root:(516,777),
    state 0x0, keycode 123 (keysym 0xbfc, righttack), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

MappingNotify event, serial 92, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 92, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 92, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

MappingNotify event, serial 92, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 123, count 1

FocusOut event, serial 96, synthetic NO, window 0x2c00001,
    mode NotifyGrab, detail NotifyNonlinear
===== End relevant part of xev output =====