bug#22983: syntax-ppss returns wrong result.

[Andreas Röhler]

On 07.09.2017 22:45, Alan Mackenzie wrote:

Hello, John.

On Tue, Sep 05, 2017 at 13:28:52 +0100, John Wiegley wrote:

Dmitry Gutov <dgutov@yandex.ru> writes:

I'm sure we want to fix design flaws. As long as there is a solid plan that
does not swap one flaw for another.

Can we have a summary of the current proposal(s) on the table? It would help
to clarify, rather than navigating past discussions. Alan has told me that
this issue is affecting people and has been outstanding for some time; I'd
like to get a better idea of its seriousness/scope, and what effect the
available solutions would have (as Dmitry says, we don't want to replace one
flaw with another).

First, I think it's worthwhile emphasising what the function purports to
do:

    syntax-ppss is a compiled Lisp function in `syntax.el'.

    (syntax-ppss &optional POS)

    Parse-Partial-Sexp State at POS, defaulting to point.
    The returned value is the same as that of `parse-partial-sexp'
    run from `point-min' to POS except that values at positions 2 and 6
    in the returned list (counting from 0) cannot be relied upon.
    Point is at POS when this function returns.

The solution I propose is to introduce a second cache into syntax-ppss,
and this cache would be used whenever (not (eq (point-min) 1)).
Whenever point-min changes, and isn't 1, this second cached would be
calculated again from scratch.

This proposal has these advantages:

(i) It would make the function deliver what its unchanged doc string
says.  This is important, given that syntax-ppss has been very widely
used within Emacs, and likely by external packages too; these will
typically have assumed the advertised behaviour of the function, without
having tested it in narrowed buffers.

(i) In the case which currently works, namely a non-narrowed buffer,
there would be only a minute slow-down (basically, there would be extra
code to check point-min and select the cache to use).

(ii) The cache for use in a narrowed buffer might well be sufficiently
fast in normal use.  If it is not, it could be enhanced readily.

I think Dmitry also proposed a method of solution some months ago,
though I don't remember in detail what it was.  Dmitry, do you still
think your solution would work?  If so, please elaborate on it.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Hi Alan and all,

assume a complex matter behind, a bunch of bugs resp. design issues, not a single one.
Fixing this would affect syntax-propertize, parse-partial-sexp, syntax-ppss and font-lock stuff at once.

http://lists.gnu.org/archive/html/emacs-devel/2016-03/msg01576.html
points at some spot. There should be more.

As a first step listing referential tests including benchmarks should be helpful.