bug#31204: 25.3; Make word motion more customizable

[Yuri Khan]

While trying to make word motion commands (Ctrl+left/right, M-f/b) more
similar to that implemented in other editors:

http://lists.gnu.org/archive/html/help-gnu-emacs/2018-04/msg00230.html

I encountered a difficulty.

The function ‘forward-word’ behaves as follows:

1. Skip to the nearest character having word constituent syntax.
2. Skip to the nearest word boundary.

Step 2, by default, finds:

* a non-word-constituent character, OR
* a transition between two adjacent characters of different scripts
  (subject to exceptions controlled by ‘word-combining-categories’ and
  ‘word-separating-categories’),

whichever comes first.

Step 2 can also be customized by modifying
‘find-word-boundary-function-table’. This enables various useful
behaviors such as ‘subword-mode’, ‘superword-mode’, and possibly CJK
word breaking rules.

Step 1, on the other hand, is not customizable at all.


The specific behavior that I was trying to implement was to find the
nearest transition:

* from a word character to a non-word character, OR
* from a non-word non-whitespace character to a word character, OR
* from a non-word non-whitespace character to a whitespace character.

As an illustration (where ‘|’ specifies word motion stops when going
left to right):

    foo| ***| +++| (|bar|)|
       ^

When cursor is after ‘foo’, step 1 of ‘forward-word’ skips to directly
before ‘bar’, missing two stops.

As a result, implementing the desired behavior requires either:

* defining separate functions ‘my-forward-word’, ‘my-backward-word’,
  ‘my-left-word’, ‘my-right-word’, ‘my-kill-word’,
  ‘my-backward-kill-word’, and possibly more, and remapping their key
  bindings; OR

* advising ‘forward-word’ with an :override.


Perhaps it would be nice to have an optional hook for step 1 of
‘forward-word’, a function that would take two arguments POS and LIMIT,
and returning the starting word boundary position from which step 2 would
then work.


In GNU Emacs 25.3.2 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2017-09-13 built on lcy01-32
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
System Description:    Ubuntu 16.04.4 LTS

Configured using:
 'configure --build=x86_64-linux-gnu --prefix=/usr
 '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var
 --disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu'
 '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
 --disable-dependency-tracking --prefix=/usr --sharedstatedir=/var/lib
 --program-suffix=25 --with-modules --with-x=yes --with-x-toolkit=gtk3
 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat
 -Werror=format-security' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_DK.utf8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 86338 5928)
 (symbols 48 19769 0)
 (miscs 40 49 121)
 (strings 32 14363 4733)
 (string-bytes 1 409522)
 (vectors 16 11755)
 (vector-slots 8 430899 3852)
 (floats 8 166 64)
 (intervals 56 231 0)
 (buffers 976 18)
 (heap 1024 33279 1050))