bug#49127: Performance degradation in encode_coding_object

[Victor Nawothnig]

Hi,

All of the following applies to Emacs 27.1 and 28.0.50.

Im currently debugging a performance degradation in haskell-mode. When editing 
code in a terminal frame, over time as I make modifications to source code, 
redrawing lines during scrolling becomes continuously slower to the point that 
it sometimes takes up 200ms for a single line to draw. This problem disappears 
once the GC runs.

With gprof/prof_events I have nailed the problem to be encode_coding_object 
looping over all markers. In degenerate cases this list can contain millions of 
markers. Traversing this list is particularly slow because of the indirection 
being a singly linked list. Based on the fact that a GC remedies this, I’m 
assuming this list contains mostly  unreachable markers. When stepping through 
encode_coding_object with GDB after a GC this list of markers shrinks to small 
double digit numbers from millions.

The source of these markers appears to be looking-at in the font locking code 
of haskell-mode, this assumption is based on the fact that commenting out the 
uses of looking-at in haskell-mode prevents the accumulation of markers and 
thus the slowdown.

One contributing factor to all of this, is that for lsp-mode to perform 
adequately, one needs a relatively high gc-cons-threshold, which means GCs that 
would clean up the markers run more rarely, leading to higher accumulation of 
markers over time.

This problem only triggers in terminal frames, but not in GUI frames. Setting 
GDB breakpoints suggests that the GUI frame never even calls into 
encode_coding_object.

So far I’m torn on whether this is a bug in the haskell-mode font locking code 
or in Emacs. What do you think?

Kind regards,
Victor