texinfmt.el + kinsoku.el combined bug

[Thierry Emery]

Hello,

First, thanks a lot for GNU Emacs 21.3, which to me is even better than
version 20.7 that i used previously!

Please find attached a bug description for the combined use of texinfmt.el
and kinsoku.el, as well as a tentative patch and Changelog.

Best regards,

Thierry Emery

-----------------------------------------------------------------------------
Used version:
 GNU Emacs 21.3.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2003-05-21

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: fr_FR
  value of $LC_CTYPE: fr_FR
  value of $LC_MESSAGES: fr_FR
  value of $LC_MONETARY: fr_FR
  value of $LC_NUMERIC: fr_FR
  value of $LC_TIME: fr_FR
  value of $LANG: fr
  locale-coding-system: iso-latin-1
  default-enable-multibyte-characters: t

Actions and symptoms:
 While installing the info part of the Japanese character input package
"SKK 12.2.0" (see http://openlab.ring.gr.jp/skk/ ), the emacs command
 `texinfo-format-buffer' fails on file "ddskk-12.2.0/doc/skk.texi" with
 message "Invalid texinfo command arg format".

Simplified scenario:
 download http://openlab.ring.gr.jp/skk/maintrunk/ddskk-12.2.0.tar.bz2
 bunzip2 -c ddskk-12.2.0.tar.bz2 | tar x ddskk-12.2.0/doc/skk.texi
 emacs -q ddskk-12.2.0/doc/skk.texi
 M-x texinfo-format-buffer

Investigation results:
 The file skk.texi contains "@footnotestyle end" and various "@footnote"
 commands.
 The "@footnote" command at point 26950 of this file is handled by
 `texinfo-format-footnote', which calls `texinfo-format-end-node', which
 calls `fill-paragraph', which calls `fill-region-as-paragraph', which itself
 calls `kinsoku', which inserts a newline between the next @samp and its
 opening brace.
 This comes from the `kinsoku' settings for "{", which has category set "<al"
 (as shown by M-x describe-categories in the unfinished skk.info buffer),
 hence `kinsoku' refuses to insert a line break after "{", and from the
 `kinsoku-shorter' logic which is willing to insert one before it.
 Of course `texinfmt' does not tolerate this, and when processing the split
 @samp command, `texinfo-parse-line-arg' signals an error with message
 "Invalid texinfo command arg format".

Tentative correction (see attached patch):
 1) modify the character category for "{" (adding ">") in the texinfmt
    working buffer,
    N.B. this is not sufficient, then `kinsoku-shorter' inserts a line break
         between "@sam" and "p{"...
 2) prevent `kinsoku' from inserting a line break inside a non-kinsoku
    (e.g. latin) word, making it skip until either a space or a character
    with category "|".

-----------------------------------------------------------------------------
--- lisp/textmodes/texinfmt.el.orig     Sun Oct 28 10:10:32 2001
+++ lisp/textmodes/texinfmt.el  Thu Aug 21 17:12:12 2003
@@ -58,6 +58,7 @@
 (require 'texnfo-upd)       ; So `texinfo-section-types-regexp' is defined.
 
 (defvar texinfo-format-syntax-table nil)
+(defvar texinfo-format-category-table nil)
 
 (defvar texinfo-vindex)
 (defvar texinfo-findex)
@@ -103,6 +104,13 @@
   (modify-syntax-entry ?} "){" texinfo-format-syntax-table)
   (modify-syntax-entry ?\' "." texinfo-format-syntax-table))
 
+;;; Category table
+
+(if texinfo-format-category-table
+    nil
+  (setq texinfo-format-category-table (copy-category-table 
(standard-category-table)))
+  ;; add that { should not be wrapped to beginning of next line
+  (modify-category-entry ?{ ?> texinfo-format-category-table))
 
 ;;; Top level buffer and region formatting functions
 
@@ -221,6 +229,7 @@
     (setq fill-column fill-column-for-info)
     ;; Install a syntax table useful for scanning command operands.
     (set-syntax-table texinfo-format-syntax-table)
+    (set-category-table texinfo-format-category-table)
 
     ;; Insert @include files so `texinfo-raise-lower-sections' can
     ;; work on them without losing track of multiple
@@ -378,6 +387,7 @@
 
     (setq fill-column fill-column-for-info)
     (set-syntax-table texinfo-format-syntax-table)
+    (set-category-table texinfo-format-category-table)
 
     (insert-buffer-substring input-buffer)
     (message "Converting %s to Info format..." (buffer-name input-buffer))
--- lisp/international/kinsoku.el.orig  Sun Jul 15 21:53:53 2001
+++ lisp/international/kinsoku.el       Thu Aug 21 18:38:20 2003
@@ -123,7 +123,11 @@
 (defun kinsoku-longer ()
   (let ((pos-and-column (save-excursion
                          (forward-char 1)
-                         (while (aref (char-category-set (following-char)) ?>)
+                         (while (and (not (eobp))
+                                     (or (aref (char-category-set 
(following-char)) ?>)
+                                         ;; protect non-kinsoku words
+                                         (not (or (eq (preceding-char) ? )
+                                                  (aref (char-category-set 
(preceding-char)) ?|)))))
                            (forward-char 1))
                          (cons (point) (current-column)))))
     (if (or (<= kinsoku-limit 0)
@@ -137,7 +141,10 @@
               (forward-char -1)
               (while (and (< linebeg (point))
                           (or (aref (char-category-set (preceding-char)) ?<)
-                              (aref (char-category-set (following-char)) ?>)))
+                              (aref (char-category-set (following-char)) ?>)
+                              ;; protect non-kinsoku words
+                              (not (or (eq (preceding-char) ? )
+                                       (aref (char-category-set 
(preceding-char)) ?|)))))
                 (forward-char -1))
               (point))))
     (if (< linebeg pos)
-----------------------------------------------------------------------------
2003-08-21  Thierry Emery <thierry.emery@club-internet.fr>

        * texinfmt.el (texinfo-format-category-table): new variable for a
          category table which is based on the standard one and adds category
          ">" to character "{".
          (texinfo-format-region, texinfo-format-buffer-1): set category
          table of working buffer to previous variable, to prevent kinsoku
          from splitting texinfo command names from their arguments.

        * kinsoku.el (kinsoku-longer, kinsoku-shorter): do not choose a line
          break position in the middle of a non-kinsoku (e.g. latin) word,
          making it skip until either a space or a character with category "|".
          (kinsoku-longer): test for end of buffer.