bug#12438: 24.2; emacs-24.2 bidi Hebrew switch introduces editing error

[jtowler]

bash-4.2$ emacs &
using M-x report-emacs-bug, bounced, and later S-D-b in gnus

C-x i xmodmap-tuple-bug-bidi-emacs-Q

file inserted at point with bug data was previously run with 
bash-4.2$ emacs -Q &

to show the editing error at the end.

Thu Sep 13 11:21:29 2012

John R. Towler, Yohanan Towler, jtowler@soncom.com

On preparing a sketch for the logical problem of all the various
keypress keys, codes, glyphs, and encoding-systems and their codes
because the solution I needed for Hebrew basic Israeli typewriter
unpointed (simplest first solution) X11 input with utf-8 wasn't
obviously done already via Xkb or Xmodmap, and the prev. solutions as
per the stds of the time might well work but are for iso-8859-8 or its
non-Unix etc., man pages for xmodmap, xsrc, x.org docs for xmodmap,
xkb, etc. all get at the problem but make logical encoding and
ordering assumptions about what the xsrcs etc. assume in their frames
for solution paths in the config files etc.: I ran into the following
problem with emacs-24.2 and bidi (Language Environment: Hebrew, input
method: hebrew) and Language Environment switching to L2R, Language
Environment: English, input method either unspecified as non l10n
specified emacs default or specified as ipa-praat which has some
utility for my interests in (formal and computational) linguistics,
other academ. areas etc., Hebrew language, other L2s, or Ln | n >=2 studied
or just as linguistic data, etc.:

file: xmodmap-tuple
on C-x f xmodmap-tuple
but here duplicated contents of xmodmap-tuple copied to
xmodmap-tuple-bug-bidi-baseline up to last typed line as described to
show what emacs does.

from bof, all is English, emacs default up until around the following point:

(the last bit is the result of several typing and editing attempts
which led to this report)

for point in buffer indicated below: C-h L RET

    English language environment

    Sample text:
      Hello!, Hi!, How are you?

    Input methods:
      english-dvorak ("DV@" in mode line)

    Character sets:
      ascii: ASCII (ISO646 IRV)

    Coding systems:
      nothing specific to English

    [back]


point is here: at C-j bol before 'collating sequence'

collating sequence, 
for hebrew unpointed, k = [א,ב,ג,ד,ה,ו,ח,ט,י,כ,ל,מ,נ,ס,ע,צ,ץ,ק,ר,ש,ת]

std emacs-24.x, here x=2 setting:

    Hebrew language environment

    Bidirectional editing is supported.

    Sample text:
      Hebrew    שלום

    Input methods (default hebrew)
      hebrew ("ע" in mode line)
      yiddish-keyman ("ײק" in mode line)
      yiddish-royal ("ײר" in mode line)
      hebrew-biblical-sil ("סִל" in mode line)
      hebrew-biblical-tiro ("תִרֹ" in mode line)
      hebrew-full ("עִ֫" in mode line)
      hebrew-lyx ("לִ" in mode line)
      hebrew-new ("ע" in mode line)

    Character sets:
      iso-8859-8: ISO/IEC 8859/8

    Coding systems:
      hebrew-iso-8bit (`8' in mode line):
            ISO 2022 based 8-bit encoding for Hebrew (MIME:ISO-8859-8).
            (alias: hebrew-iso-8bit iso-8859-8 iso-8859-8-e iso-8859-8-i)
      windows-1255 (`h' in mode line):
            windows-1255 (Hebrew) encoding (MIME: WINDOWS-1255)
            (alias: windows-1255 cp1255)
      cp862 (`D' in mode line):
            DOS codepage 862 (Hebrew)
            (alias: cp862 ibm862)

    [back]
C-u C-x = on the aleph of the sequence/set typed in the orig. file

             position: 3037 of 5940 (51%), column: 27
            character: א‎ (displayed as א‎) (codepoint 1488, #o2720, #x5d0)
    preferred charset: iso-8859-8 (ISO/IEC 8859/8)
code point in charset: 0xE0
               syntax: w        which means: word
             category: .:Base, R:Right-to-left (strong)
             to input: type "t" with hebrew
          buffer code: #xD7 #x90
            file code: #xD7 #x90 (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    x:-misc-fixed-medium-r-normal--15-140-75-75-c-90-iso8859-8 (#xE0)

Character code properties: customize what to show
  name: HEBREW LETTER ALEF
  general-category: Lo (Letter, Other)
  decomposition: (1488) ('א')


switching via C-x RET L Hebrew, C-x RET C-\ hebrew works correctly to
type the Heb. alph. with R2L correctly handled, all of this is
unshifted keyboard keys, knowing the punct keys etc. were not typed.

C-x RET L English, C-x RET C-\ ipa-praat

    %% prev. experiments have shown that switching back to Language
    %% Environment: English, input-method unchanged results in L2R
    %% Hebrew chars for qwerty keys, and I don't know the Dvorak
    %% keyboard, and Praat is a very useful libre portable acoustic
    %% phonetics laboratory base, and some version of IPA is most
    %% useful, so why not ipa-praat

lc-keys mapped to punct other than qwerty ascii = [\,, \. , \q,  \' ]

The escaped glyphs in the sequence/set are the result of switching
between Language Environment:Hebrew, input-method: hebrew to type the
remaining unshifted qwerty keys from z to /, a to ', q to p and back
to Language Environment:English, input-method:ipa-praat on a pc10[145]
laptop keyboard set to swap ctrl and caps-lock so the broken pc
keyboard keys are in the right place.

C-x RET L English, C-x RET C-\ ipa-praat

punct = [\?, \,, \., \[, \], \\, \|, \{, \}, \'\', \', \`, \~, \!, \@, \#, \$, 
\%, \^, \&, \*, \(, \), \-, \=, \+, \_, \maqqeph, \;, \:, \"]

num=[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]


access-feature-val-pr(keybd-array(key), print-name(glyph))

qwerty unshifted etc. bottom row of keys to m with switch to Language 
Environment:Hebrew, input-mode:hebrew for the corresponding hebrew glyph

[(z, ז), (x, ס), (c, ב), (v, ה), (b, נ), (n, מ), (m, צ), 

next few keys across the bottom of the keyboard switching to
Hebrew:hebrew for the key press of each key until the error appears,
using C-k on the row above to save typing and then typing after the
mem rparen: , (, , C-x RET L Hebrew C-x RET C-\ hebrew keypress qwerty
, for the expected Hebrew glyph

[(z, ז), (x, ס), (c, ב), (v, ה), (b, נ), (n, מ), (m, צ), (, , ת
EOF

------
just in case:
bash-4.2$ uname -a

NetBSD peano.jtcl.org 6.0_BETA2 NetBSD 6.0_BETA2 (Peano3 -r netbsd-6 -D 
8/10/2012 -current auich* bind* ) #0: Thu Aug 23 11:56:58 UTC 2012  
root@peano.jtcl.org:/usr/obj/sys/arch/i386/compile/Peano3 i386

---


In GNU Emacs 24.2.1 (i386-unknown-netbsdelf6.0., X toolkit, Xaw3d scroll bars)
 of 2012-08-28 on peano.jtcl.org
Windowing system distributor `The Xorg Foundation', version 11.0.11003000
Configured using:
 `configure '--x-includes=/usr/X11R7/include'
 '--x-libraries=/usr/X11R7/lib' '--enable-asserts' '--enable-checking'
 '--enable-profiling' '--enable-dependency-tracking'
 '--without-mail-unlink' '--with-sound' '--with-x-toolkit=lucid'
 '--with-wide-int' '--without-xpm' '--without-jpeg' '--without-tiff'
 '--without-gif' '--without-png' '--without-rsvg'
 '--without-imagemagick' '--without-gpm' '--without-dbus'
 '--without-gconf' '--without-gsettings' '--without-selinux' '--with-x'
 '--without-xft' 'LDFLAGS=-L/usr/X11R7/lib -L/usr/local/lib'
 'CPPFLAGS=-I/usr/X11R7/include -I/usr/local/include''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Info

Minor modes in effect:
  display-time-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
q C-h i m e m a c s <return> m e m a c s <return> C-g 
m B u g s <return> SPC SPC SPC SPC SPC SPC SPC SPC 
<backspace> ` x C-g <down-mouse-1> <mouse-1> <escape> 
x r e p o r t - e m a c s - b u g <return>

Recent messages:
Loading /usr/local/home/jtowler/elisp/haskell-mode-2.8.0/haskell-site-file.el 
(source)...done
Loading ispell...done
Loading timeclock...done
Preparing diary...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Composing main Info directory...done
Making completion list...
Quit
Quit

Load-path shadows:
/usr/local/home/jtowler/elisp/ispell hides 
/usr/local/share/emacs/24.2/lisp/textmodes/ispell
/usr/local/share/emacs/site-lisp/esh-mode hides 
/usr/local/share/emacs/24.2/lisp/eshell/esh-mode

Features:
(shadow sort gnus-util mail-extr warnings emacsbug message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils help-mode jka-compr info view cal-china
lunar cal-bahai holidays hol-loaddefs cal-julian solar cal-dst diary-lib
cal-hebrew diary-loaddefs mule-util cal-move cal-menu calendar
cal-loaddefs time timeclock ispell preview-latex tex-site auto-loads
bbdb-autoloads bbdb timezone printing ps-print ps-def lpr slime byte-opt
bytecomp byte-compile cconv macroexp derived help-fns edmacro kmacro
easy-mmode easymenu pp comint regexp-opt ansi-color ring hyperspec
thingatpt browse-url cl time-stamp time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dynamic-setting
x-toolkit x multi-tty emacs)