Re: The current state of the comment-cache branch

[Alan Mackenzie]

Hello, Stefan.

Thanks for such a thorough summary of the issue.

On Sun, Dec 25, 2016 at 11:07:46AM -0500, Stefan Monnier wrote:
> > Also, I wonder why do we need all these changes in syntax.c, when the
> > problem is AFAIK only with C mode and its derivatives.  Are these
> > changes beneficial in any way to modes with non-C syntax?

The actual cause of the problem is in syntax.c.

> The core of the problem is calls to (forward-comment -1).

More precisely, in CC Mode, what triggers the problem is usually
scan-lists with a negative `count' argument.  This calls back_comment,
which is where the problem happens.

> Both this comment-cache and my syntax-ppss patch attack this specific
> operation only.  This operation is currently implemented by trying to
> skip comments "locally" by parsing backward, but when that fails, we use
> parse-partial-sexp from point-min to find the matching opening of
> a comment closer.

> This expensive forward pass is not a problem in practice unless your buffer
> is huge, or unless it happens many times within a single command.
> These can happen in any major mode, basically.
> It's only a historical accident if it seems to affect CC-mode more.

> open-paren-in-column-0-is-defun-start is the current hack to try and
> reduce the occurrence of this problem by only calling parse-partial-sexp
> from the closest open-paren-in-column-0, which works fairly well in
> practice for typical Elisp code as well as for typical C code (tho not
> for all coding styles).

> [ AFAIK open-paren-in-column-0-is-defun-start was specifically designed
> for this (forward-comment -1) issue, but it is also used in
> beginning-of-defun, although the two uses are completely independent and
> of a different nature: the use in forward-comment is supposed to be only
> an optimization, whereas the use in beginning-of-defun is meant to
> really change the result.  So beginning-of-defun is completely
> irrelevant to and independent from this thread.  ]

> Also in the latest case where a major slow down showed up in CC-mode,
> the problem was reversed: CC-mode had some special code for when
> open-paren-in-column-0-is-defun-start is non-nil (can't remember what
> it was for), which was the cause of the slow down, IIRC.

Bug #22884.  This isn't quite accurate.  The slowdown happened when C
Mode thought that the "(" in the comment was a BOD and in repeated
scanning to point (which is typically harmless) was scanning many times
over ~40k characters.

> Both Alan's comment-cache and my syntax-ppss patch aim to replace the
> open-paren-in-column-0-is-defun-start hack so as to completely eliminate
> this pathological (forward-comment -1) case.

Is your syntax-ppss patch at a state where it could be benchmarked and
tested?  If so, please say again where it is, or put it somewhere
public.  Dmitry is interested in it too.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).