problem with comint-carriage-motion and inhibit-eol-conversion

[Noah Friedman]

The current definition of comint-carriage-motion does not work in certain
cases when inhibit-eol-conversion is set to t.

There are (IMO) badly-behaved programs that output a CR at the end of an
output chunk instead of at the beginning; for example `rsync --progress'
does this.  I've written the following simulator of the output:

    #!/usr/bin/perl

    my $i = 0;
    my $step = 1024;
    my $total = $step * 50;

    select (STDOUT);
    $| = 1;

    while ($i <= $total)
      {
        printf "%8d (%d%%)\r", $i, (100 * $i / $total);
        $i += $step;
        select (undef, undef, undef, 0.5);
      }
    print "\n";

You can inspect the output chunks as they arrive to emacs with the
following:

    (progn
      (setq output-chunks nil)
      (defun accum-output-chunks (s)
        (setq output-chunks (cons s output-chunks)))
      (add-hook 'comint-output-filter-functions 'accum-output-chunks))

then running the simulator above and looking at the value of output-chunks.

Now, if inhibit-eol-conversion is set to nil before starting a shell
process (the default), then for some reason--despite the fact that the CR
is output at the end of each output segment instead of at the
beginning--emacs doesn't see the CR until the start of the following output
chunk.  Therefore comint-carriage-motion works just fine.

However, if you set inhibit-eol-conversion to t before starting a shell
process, then the output chunks are received by emacs exactly like they
are sent by the simulator, i.e. the CRs appear at the end of each output
chunk.

The consequence of this is that the current implementation of
comint-carriage-motion will not handle "overwriting", and the output just
accumulates in the buffer like so:

       0 (0%)    1024 (2%)    2048 (4%)    3072 (6%)    4096 (8%)    [etc]

I have a solution to this, but it means that sometimes a bare ^M will be
visible in the buffer.  I think this is a worthwhile tradeoff, and it only
happens when inhibit-eol-conversion is t.

Here's my rewrite of comint-carriage-motion.  There's several functional
behavioral changes I made here:

       * Make sure not to modify pending input (that may have quoted
         carriage returns in it; who knows.)

       * Ignore CR at very end of output chunk (if any); reprocess when
         next output chunk arrives.

       * Don't try to backspace past narrowing (unrelated, but I
         encoutered this when erasing a buffer while output occured).


(defun comint-carriage-motion (start end)
  "Interpret carriage control characters in the region from START to END.
Translate carriage return/linefeed sequences to linefeeds.
Make single carriage returns delete to the beginning of the line.
Make backspaces delete the previous character."
  (unless (= start end) ; ignore empty output
    (save-excursion
      (save-restriction
        (let ((inhibit-field-text-motion t)
              (buffer-read-only nil))

          ;; widen to get bol, but keep end
          ;; we don't want to modify pending input
          (widen)
          (goto-char start)
          (beginning-of-line)
          ;; Note that we don't search the last char in the output chunk
          ;; for CR handling because it might be a bare CR, and some
          ;; programs like rsync with `--progress' output a CR at the end
          ;; of each output chunk instead of at the beginning of the next
          ;; one.  Ignoring the last char means a naked CR is left visible
          ;; in the buffer, but on the next pass it will be treated
          ;; properly.  Otherwise, the buffer accumulates output that was
          ;; intended to be overwritten.
          ;; This happens mainly when inhibit-eol-conversion is t.
          (narrow-to-region (point) (1- end))
          (save-match-data
            ;; CR LF -> LF
            ;; Note that this won't work properly when the CR and LF
            ;; are in different output chunks, but this is probably an
            ;; exceedingly rare case (because they are generally
            ;; written as a unit), and to delay interpretation of a
            ;; trailing CR in a chunk would result in odd interactive
            ;; behavior (and this case is probably far more common).
            ;; There may be more than one CR if onlcr is set.
            (while (re-search-forward "\r+$" nil t)
              (delete-region (match-beginning 0) (match-end 0)))
            ;; bare CR -> delete preceding line
            (goto-char (point-min))
            (while (search-forward "\r" nil t)
              (delete-region (point) (line-beginning-position)))
            ;; BS -> delete preceding character
            (goto-char (point-min))
            ;; extend region to include final char in output chunk
            (narrow-to-region (point-min) (1+ (point-max)))
            (while (search-forward "\b" nil t)
              (and (> (- (point) (point-min)) 1)
                   (delete-char -2)))))))))