bug#58558: 29.0.50; re-search-forward is slow in some buffers

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> IIUC a summary of what we know so far:
> - the "yant/re" benchmark is ~20x slower in Emacs-29 than in Emacs-28.
> - removing all text properties reduces the factor down to about ~15x.
> - that difference is absent after a fresh start: it only appears over time.
>
> Since this benchmark always matches the same regexp, I can't imagine how
> the regexp cache could thrash, so it definitely seems to come from
> something else.
>
> I'd curious to know the result of the following tests:
>
> - Run the same benchmark twice in a row: does the second run take the
>   same time as the first, or is the second run significantly faster?
>   [ if it's faster it might be due to something like the on-the-fly
>     `syntax-propertize`ation.

After 11 hours of Emacs uptime and some edits in the buffer (actually,
just a few hours; mostly idle), running the benchmark-progn
repetitively:

;; Elapsed time: 8.339753s
;; Elapsed time: 9.243140s
;; Elapsed time: 9.868761s
;; Elapsed time: 10.330362s
;; Elapsed time: 11.279218s
;; Elapsed time: 13.581893s
;; Elapsed time: 13.675609s
;; Elapsed time: 14.553157s
;; Elapsed time: 14.651782s
;; Elapsed time: 17.253983s

The elapsed time gradually increases. It is definitely a clue, but very
odd one.

>     BTW, what does the profiler-start/report say?
>     Is the time 100% spent in `re-search-forward`?  ]

;; w CPU profiler
;; Elapsed time: 19.628828s

;; profiler:
;; 19954  99% - command-execute
;; 19926  99%  - funcall-interactively
;; 19627  98%   - eval-expression
;; 19627  98%    - let
;; 19627  98%     - progn
;; 19627  98%        while
;; ------------ no more data inside while ---------

Nothing useful. It's like while loop is doing something bad, but how so
in (benchmark-progn (while (re-search-forward yant/re nil t))) ??

I also tried find-file-literally and the timing gets back to fresh Emacs
(even faster):

;; find-file-literally
;; Elapsed time: 0.592935s

Then, I re-opened the file normally.

;; re-open the file
;; Elapsed time: 7.348727s

Note how the time is back to 7-8 seconds, but not same as fresh Emacs.

> - Try to reduce the number of "features" used in the regexp to see how
>   it affects the slow down.  Maybe try a "binary search" where you try
>   to reduce the regexp to something much simpler and see if some regexps
>   exhibit the slowdown while others don't?

Hmm. I tried a very simple regexp "^\\*+ " 10 times in a row:

;; Elapsed time: 0.267681s
;; Elapsed time: 0.381607s
;; Elapsed time: 0.342378s
;; Elapsed time: 0.350618s
;; Elapsed time: 0.376871s
;; Elapsed time: 0.446346s
;; Elapsed time: 0.472543s
;; Elapsed time: 0.529925s
;; Elapsed time: 0.604101s
;; Elapsed time: 0.665601s

It is generally faster, but still relatively slow and gets worse over
time.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>