bug#58558: 29.0.50; re-search-forward is slow in some buffers

[Stefan Monnier]

>> Huh, very odd.  Almost as something is...  fragmenting in the buffer?
>> We do have many caches and stuff -- perhaps something is...  degrading?
>> 
>> I guess some C-level perf measurements would be handy here, but that's
>> not something I know much about.  Anybody?
>
> AFAIU, we use elaborate caching for regular expressions, so maybe that
> is related.  Stefan, any ideas?

The regexp cache hasn't changed between 28 and 29, so that seems
unlikely to be the source of the problem.  But that cache is fairly
simple-minded, so it's possible that for some reason it thrashes in
Emacs-29 but not in Emacs-28 (but see below).

IIUC a summary of what we know so far:
- the "yant/re" benchmark is ~20x slower in Emacs-29 than in Emacs-28.
- removing all text properties reduces the factor down to about ~15x.
- that difference is absent after a fresh start: it only appears over time.

Since this benchmark always matches the same regexp, I can't imagine how
the regexp cache could thrash, so it definitely seems to come from
something else.

I'd curious to know the result of the following tests:

- Run the same benchmark twice in a row: does the second run take the
  same time as the first, or is the second run significantly faster?
  [ if it's faster it might be due to something like the on-the-fly
    `syntax-propertize`ation.
    BTW, what does the profiler-start/report say?
    Is the time 100% spent in `re-search-forward`?  ]
- Try to reduce the number of "features" used in the regexp to see how
  it affects the slow down.  Maybe try a "binary search" where you try
  to reduce the regexp to something much simpler and see if some regexps
  exhibit the slowdown while others don't?


        Stefan