Re: [PATCH] Bug: fragile org refile cache

[Maxim Nikulin]

On 29/04/2021 23:08, Ihor Radchenko wrote:
> Did you do any benchmarks? I just tried
>
> Outline path without cache:

I have expanded your tests to make them more close to org-get-outline-path

org-get-outline-path without cache
| 5.459114059 | 12 | 1.2053580009999987 |
org-get-outline-path with cache
| 2.166378543 |  5 | 0.5011187770000003 |

Custom code to track outline path during scan
| 0.718288205 | 4 | 0.4005034349999992 |
The same without (org-trim ...) stuff
| 0.208798040 | 0 |                0.0 |

From my point of view avoiding org-get-outline-path allows to 
significantly improve performance. Removing cookies, etc. is far from 
being negligible.

Test file is org-manual.org with 404 headings. Since it is not so much, 
I set minimal CPU frequency.

Notice that outline cache is cleared before each scan to simulate the 
case when org-goto is invoked just after emacs start or after 
adding/removing some lines in the beginning of the file. Actually it is 
a problem of outline cache that it easily became stale due to e.g. extra 
line close to buffer start.

Details:

#+begin_example bash
for i in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor ;
do
    echo powersave >"$i" ;
done
#+end_example

#+begin_src elisp
  (byte-compile-file (expand-file-name "~/src/emacs/org-mode/lisp/org.el"))
  (defun nm-count-headings (headings)
    (let ((n 0))
      (dolist (h headings)
        (setq n (+ n (length h))))
      (list (length headings) n)))
#+end_src

#+RESULTS:
: nm-count-headings

#+begin_src elisp
  (defun ir-tst-scan (buffer use-cache)
    (with-current-buffer buffer
      (goto-char 1)
      (let ((result nil))
        (while (re-search-forward "^\\*+[ \t]" nil t)
          (beginning-of-line)
          (let ((case-fold-search nil))
            (looking-at org-complex-heading-regexp))
          (push (org-get-outline-path t use-cache) result)
          (end-of-line))
        result)))
  (byte-compile 'ir-tst-scan)
  (mapcar
   (lambda (use-cache)
     (let ((headings))
       (append
        (list use-cache)
        (benchmark-run 10
          (progn
            (setq org-outline-path-cache nil)
            (setq headings
                  (ir-tst-scan "org-manual.org" use-cache))))
        (nm-count-headings headings))))
   '(nil t))
#+end_src

#+RESULTS:
| nil | 5.459114059 | 12 | 1.2053580009999987 | 404 | 1069 |
| t   | 2.166378543 |  5 | 0.5011187770000003 | 404 | 1069 |

| use-cache | benchmark | |                   | headings | outline 
components |

#+begin_src elisp
  (defun nm-tst-scan (buffer clean-headers)
    (with-current-buffer buffer
      (goto-char 1)
      (let ((result nil))
        (let ((outline-path))
          (while (re-search-forward "^\\*+[ \t]" nil t)
            (beginning-of-line)
            (let ((case-fold-search nil))
              (looking-at org-complex-heading-regexp))
            (let ((heading (match-string-no-properties 4))
                  (heading-level (length (match-string-no-properties 1))))
              (while (and outline-path (<= heading-level (caar outline-path)))
                (pop outline-path))
              (push (cons heading-level
                          (if (not heading)
                              ""
                            (if (not clean-headers)
                                heading
                              (org-trim
                               (org-link-display-format
                                (replace-regexp-in-string
                                 "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
                                 heading))))))
                    outline-path))
            (end-of-line)
            (push (mapcar #'cdr outline-path) result)))
        result)))
  (byte-compile 'nm-tst-scan)
  (mapcar
   (lambda (clean-headers)
     (let ((headings))
       (append
        (list clean-headers)
        (benchmark-run 10
          (progn
            (setq org-outline-path-cache nil)
            (setq headings
                  (nm-tst-scan "org-manual.org" clean-headers))))
        (nm-count-headings headings))))
   '(nil t))
#+end_src

#+RESULTS:
| nil | 0.20879804000000002 | 0 |                0.0 | 404 | 1069 |
| t   |         0.718288205 | 4 | 0.4005034349999992 | 404 | 1069 |

| clean-headers | benchmark |   |                    | headings | 
outline components |