Opportunistic GC

[Stefan Monnier]

I've been running with the code below recently to try and see if
opportunistic GC can be worth the trouble.

I've mostly focused on measuring the GC behavior rather than on tuning
the opportunistic parameters, so there's no doubt we can do better, but
here's what I've gotten so far:

- Gnus-focused session:
  ((opportunistic-gcs . 848)
   (jit 6 0 0) (cmd 526 241 707) (noncmd 92 1 2)
   (commands . 65745))
- General session:
  ((opportunistic-gcs . 1332)
   (jit 115 0 0) (cmd 239 19 89) (noncmd 1749 18 38)
   (commands . 176431))

The precise meaning of those numbers can be seen in the code, but here's
the explanation in words:
`commands` is the number of commands that were run.
`opportunistic-gcs` is the number of times we performed GC opportunistically
`jit`, `cmd`, and `noncmd` are the number of times we GC'd non-opportunistically
during a `cmd`, between commands (for `noncmd`) or during jit-lock.
The first number is the number of times a single GC took place
during/between the command, the second number counts the number of times
several GCs took place, and the third is to sum of those "multiple GCs".

The middle and third numbers are those for which opportunistic GC
fundamentally can't help very much: the command (or the work done
between commands) was just sufficiently long-running that it's hard to
avoid running a GC during this period.

[ Note that the way we measure for jit-lock it's really unlikely we'll
  ever get "multiple GCs" because it would require >1 GC performed
  during a single jit-lock chunk.  ]

Note that because my opportunistic GC calls (garbage-collect-maybe 3),
every opportunistic GC avoids only 1/3 non-opportunistic GC.

So, while these numbers don't look completely hopeless, they seem to
indicate that opportunistic GCs aren't very useful at least with the
tuning parameters I used.  To be more useful, maybe we'd need to
increase the non-opportunistic `gc-cons-percentage` and/or reduce the
idle-time before opportunistic GC (see "magic formula" below).

There's also a question of where those `noncmd` GCs come from.
(timers?  process filters?  non-jit part of redisplay? ...?)

        Stefan


(defvar gc--opportunistic-last-gcs gcs-done)
(defvar gc--opportunistic-state 'noncmd)
(defvar gc--opportunistic-counters nil)

(defun gc--check ()
  (let ((gcs-counted
         (+ (alist-get 'multi-gcs gc--opportunistic-counters 0)
                   (alist-get 'earlier-gcs gc--opportunistic-counters 0)
                   (alist-get 'single-gc-cmds gc--opportunistic-counters 0)
                   (alist-get 'noncmds-gcs gc--opportunistic-counters 0)
                   (alist-get 'opportunistic-gcs gc--opportunistic-counters 0)
                   (or (car (alist-get 'error-gcs gc--opportunistic-counters)) 
0))))
    (unless (= gcs-done gcs-counted)
      (push (+ (- gcs-done gcs-counted)
               (or (car (alist-get 'error-gcs gc--opportunistic-counters)) 0))
            (alist-get 'error-gcs gc--opportunistic-counters)))))

(defun gc--opportunistic-record (nextstate)
  (let ((counts (alist-get gc--opportunistic-state gc--opportunistic-counters)))
    (unless counts
      (setf (alist-get gc--opportunistic-state gc--opportunistic-counters)
            (setq counts (list 0 0 0))))
    (pcase (prog1 (- gcs-done gc--opportunistic-last-gcs)
             (setq gc--opportunistic-last-gcs gcs-done))
      ((pred (>= 0)) nil)
      (1 (cl-incf (nth 0 counts)))
      (gcs (cl-incf (nth 1 counts))
           (cl-incf (nth 2 counts) gcs))))
  (setq gc--opportunistic-state nextstate))

(defun gc--opportunistic-postch ()
  (cl-incf (alist-get 'commands gc--opportunistic-counters 0))
  (gc--opportunistic-record 'noncmd))

(defun gc--opportunistic-prech ()
  (cl-callf identity
      (alist-get 'earlier-gcs gc--opportunistic-counters gcs-done))
  (gc--opportunistic-record 'cmd)
  ;; (gc--check)
  )

(defun gc--opportunistic-jitlock (orig-fun start)
  (if (eq gc--opportunistic-state 'cmd)
      ;; Count jit-lock execution which happens during a command as
      ;; being part of command execution rather than as part of jit-lock!
      (funcall orig-fun start)
    (let ((gc--opportunistic-state gc--opportunistic-state))
      (gc--opportunistic-record 'jit)
      (unwind-protect
          (funcall orig-fun start)
        (gc--opportunistic-record 'postjit)))))

(add-hook 'pre-command-hook #'gc--opportunistic-prech)
(add-hook 'post-command-hook #'gc--opportunistic-postch)
(advice-add 'jit-lock-function :around #'gc--opportunistic-jitlock)

(defun gc--opportunistic ()
  "Run the GC during idle time."
  ;; This is good for two reasons:
  ;; - It reduces the number of times we have to GC in the middle of
  ;;   an operation.
  ;; - It means we GC when the C stack is short, reducing the risk of false
  ;;   positives from the conservative stack scanning.
  (when (garbage-collect-maybe 3)
    (cl-incf (alist-get 'opportunistic-gcs gc--opportunistic-counters 0))
    ;; Don't double count this GC in other categories.
    (cl-incf gc--opportunistic-last-gcs)
    ;; Recalibrate the timer.
    (cancel-function-timers #'gc--opportunistic)
    (run-with-idle-timer
     ;; FIXME: Magic formula!
     (+ 1 (* 10 (/ gc-elapsed gcs-done))) t #'gc--opportunistic)))

(defun gc--opportunistic-score ()
  "Show the current counters's that keep track of GC behavior."
  (interactive)
  (message "%S" gc--opportunistic-counters))