bug#10245: 23.3; handle Common Lisp :documentation strings

[Robert Brown]

Several Common Lisp forms, such as defgeneric and defclass, support
documentation strings following a :documentation keyword.

Here's an example of a point class with documentation strings:

  (defclass foo ()
    ((x :reader x :initarg :x :documentation "X coordinate")
     (y :reader y :initarg :y :documentation "Y coordinate"))
    (:documentation "A point in the plane"))

Emacs currently fontifies the documentation strings above using
font-lock-string-face instead of font-lock-doc-face.

The code below fixes the problem for me.  It is a refactoring of
lisp-font-lock-syntactic-face-function in lisp/emacs-lisp/lisp-mode.el
with the addition of a new function to detect documentation keywords.
If lisp-font-lock-syntactic-face-function is replaced with the code
below, Emacs fontifies strings after :documentation keywords using
font-lock-doc-face.

(defun string-in-doc-position-p (state)
  (let* ((listbeg (nth 1 state))
         (firstsym (and listbeg
                        (save-excursion
                          (goto-char listbeg)
                          (and (looking-at "([ \t\n]*\\(\\(\\sw\\|\\s_\\)+\\)")
                               (match-string 1)))))
         (docelt (and firstsym (get (intern-soft firstsym)
                                    lisp-doc-string-elt-property))))
    (and docelt
         ;; It's a string in a form that can have a docstring.
         ;; Check whether it's in docstring position.
         (save-excursion
           (when (functionp docelt)
             (goto-char (match-end 1))
             (setq docelt (funcall docelt)))
           (goto-char listbeg)
           (forward-char 1)
           (condition-case nil
               (while (and (> docelt 0) (< (point) startpos)
                           (progn (forward-sexp 1) t))
                 (setq docelt (1- docelt)))
             (error nil))
           (and (zerop docelt) (<= (point) startpos)
                (progn (forward-comment (point-max)) t)
                (= (point) (nth 8 state)))))))

(defun string-after-documentation-keyword-p (state)
  (and (nth 1 state)                    ; we are inside a Lisp form
       (let ((startpos (nth 8 state)))
         (and startpos
              (let ((ok nil)
                    (prevsym (save-excursion
                               (goto-char startpos)
                               (condition-case nil
                                   (progn (backward-sexp 1) (setf ok t))
                                 (error nil))
                               (and ok
                                    (looking-at "\\(\\sw+\\)")
                                    (match-string 1)))))
                (equal prevsym ":documentation"))))))

(defun lisp-font-lock-syntactic-face-function (state)
  (if (nth 3 state)
      ;; This might be a (doc)string or a |...| symbol.
      (let ((startpos (nth 8 state)))
        (if (eq (char-after startpos) ?|)
            ;; This is not a string, but a |...| symbol.
            nil
          (if (or (string-in-doc-position-p state)
                  (string-after-documentation-keyword-p state))
              font-lock-doc-face
            font-lock-string-face)))
    font-lock-comment-face))



In GNU Emacs 23.3.1 (x86_64-unknown-linux-gnu)
 of 2011-07-15 on paradicsom.nyc.corp.google.com
configured using `configure  '--prefix' '/local/software/package/emacs-23.3.1' 
'--without-x''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
ESC [ > 1 ; 2 3 0 5 ; 0 c ESC x b u g - TAB ESC DEL 
ESC DEL r e p o r t - TAB RET

Recent messages:
("emacs")
Loading /local/software/source-trees/emacs-jabber/jabber-autoloads.el 
(source)...done

Load-path shadows:
/local/software/source-trees/cc-mode/cc-fonts hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-fonts
/local/software/source-trees/cc-mode/cc-align hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-align
/local/software/source-trees/cc-mode/cc-menus hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-menus
/local/software/source-trees/cc-mode/cc-engine hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-engine
/local/software/source-trees/cc-mode/cc-mode hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-mode
/local/software/source-trees/cc-mode/cc-awk hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-awk
/local/software/source-trees/cc-mode/cc-defs hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-defs
/local/software/source-trees/cc-mode/cc-vars hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-vars
/local/software/source-trees/cc-mode/cc-styles hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-styles
/local/software/source-trees/cc-mode/cc-bytecomp hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-bytecomp
/local/software/source-trees/cc-mode/cc-compat hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-compat
/local/software/source-trees/cc-mode/cc-cmds hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-cmds
/local/software/source-trees/cc-mode/cc-langs hides 
/local/software/package/emacs-23.3.1/share/emacs/23.3/lisp/progmodes/cc-langs

Features:
(shadow sort mail-extr message ecomplete rfc822 mml easymenu mml-sec
password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231
rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc
time-date mm-util mail-prsvr gmm-utils wid-edit mailheader canlock sha1
hex-util hashcash mail-utils warnings emacsbug uniquify advice help-fns
advice-preload slime-autoloads regexp-opt cl cl-19 ediff-hook vc-hooks
lisp-float-type lisp-mode register page menu-bar rfn-eshadow timer
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
loaddefs button minibuffer faces cus-face files text-properties overlay
md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process multi-tty emacs)