Re: Bug #25608 and the comment-cache branch

[Alan Mackenzie]

Hello, Dmitry.

On Mon, Feb 06, 2017 at 04:09:42 +0200, Dmitry Gutov wrote:
> On 04.02.2017 12:24, Alan Mackenzie wrote:

> > You want comment-cache to be wholly abandoned.

> At least the part that maintains a separate cache. I'm not sure if 
> there's anything else there.

The essence of comment-cache is scanning comments only in the forward
direction.  This is impractical without a good cache.  The syntax-ppss
cache is wholly inadequate here (and would be even if it worked in the
general case).

> >> And then we should seek the simplest solution that satisfies all of our
> >> requirements.

> > As simple as possible, but definitely not simpler.  The "solution" you
> > favour is too simple.  It doesn't work all the time.

> I concede it's not ideal. However, I strongly believe "fixing" the 
> narrowing problem in syntax-ppss with take care of this example, *and* 
> will result in lower overall complexity and maintenance burden.

There's no sign of syntax-ppss being fixed.  Bug #22983 has been open
for almost a year, and despite repeated requests from me, there has been
no movement on it.

Anyways, there are other problems with the "alternative patch".  It
doesn't clear it's caches when syntax-table properties are applied to or
removed from a buffer.  It doesn't clear its caches when a "literal
relevant" change is made to the current syntax table, or a different
syntax-table is made current.  comment-cache handles these situations
correctly - that's where its perceived complexity scores.

> Consider the problems you've had merging master into the comment-cache 
> branch. If there were conflicts, that means the new code touches a 
> changing area, and it will need to be considered and taken care of by 
> the maintainers, probably on an ongoing basis.

comment-cache has rewriten backward_comment entirely, hence the
troublesome merge.  It's no more difficult for maintainers than the
current version of Emacs.

> The AP, on the other hand, still applies cleanly.

Not surprisingly.  It's simplistic, too simplistic.

> >> "It introduces a second source of truth" seems like a concise summary.

> > So what?  There are any number of "sources of truth" in Emacs.  If one
> > of them turns out to be a "source of untruth" we call that a bug, and we
> > fix it.

> One normally adds an alternative source of truth (i.e. a "cache") to fix 
> a significant performance problem, when one really can't do so otherwise.

So far, there's no fully satisfactory alternative to comment-cache on
the table.

> It seems we agree now that comment-cache's existence can't be justified 
> by performance considerations.

> Cache invalidation is a known hard problem in CS, so we generally don't 
> want to have extra caches.

It might be a difficult problem but it's not NP-complete, or anything
like that.  comment-cache solves the cache invalidation.  syntax-ppss,
used in the "alternative patch" doesn't.  (See above.)

> >> At best, it'll use more memory than it has to.

> > The thing to do here is measure this extra memory.  I did this back in
> > spring last year, and the number of extra conses used for the cache was
> > not inordinately high.  Especially not for a 64-bit machine with several
> > gigabytes of RAM.

> Maybe it's not bad, without a direct link it's hard for me to comment on 
> that now. But "no extra memory usage" would be a better outcome anyway.

It would, but nobody's come up with a satisfactory way to achieve this.

> > I think you're seeing something that's not there.  You're picturing some
> > imagined process where two alternative ways of storing information have
> > great difficulty staying together, and somehow, over time, are destined
> > to drift apart.  Sort of like two national currencies trying to stay
> > pegged to eachother, or something like that.

> I'm picturing weird syntax highlighting/defun navigation/etc behavior 
> that comes and goes seemingly randomly, and which forces us to debug 
> both cache mechanisms to see which one is getting something wrong.

Oh, I've had plenty of practice at this sort of thing.  Open parens at
column 0 in comments have been a frequent trigger for these problems.
comment-cache's cache is simple, and should thus be easy to verify.

> They don't even have to drift far apart functionality-wise, as long as 
> their implementations are largely independent.

They shouldn't drift apart at all.  But drifting apart is no worse a
problem than a single cache being wrong.

[ .... ]

> > Note, in this context, that syntax-ppss is broken (bug #22983) and
> > doesn't look like getting fixed any time soon, yet the world hasn't come
> > to an end.

> A consistently "wrong" behavior is better than having some standard 
> library functions work "correctly", and some otherwise.

A consistently wrong behaviour in a cache handler is not better.

> Consider this again: as long as syntax-ppss continues to have problems 
> in the cases you imagine, the caches _will_ diverge in those cases.

Yes they will.  In those cases, it would still be better if
backward_comment functioned correctly.

> Honestly, my head hurts when I start thinking up problem examples, but 
> I'm sure the users and authors of modes that define 
> syntax-propertize-function and/or use syntax-ppss won't like them.

They won't see them.

> >>> Note that there has been NO constructive criticism of comment-cache.

> >> That's insulting, Alan.

> > It might be, but I think it's true.  You want comment-cache to be wholly
> > abandoned.  You are not suggesting ways to make it better.  You haven't
> > tried it, that I'm aware of.  You haven't looked for flaws, with the
> > intention of getting them fixed.

> You seem to argue that a high-level criticism can't be constructive, and 
> that any good one has to discuss lower-level implementation details.

Arguing for complete abandonment is not constructive criticism.

>  > Instead you are putting forward
>  > reasons, not all of them good, for abandoning comment-cache.

> Aside from "two sources of truth", the other reason is that we have a 
> much-simpler patch that gives us (or will eventually give) the same 
> benefits.

It doesn't.  It doesn't clear its caches when it ought to because of
changes in syntax-table text properties, changes in the current syntax
table, or swapping to a different syntax table.  comment-cache handles
all of these things.

I'm not saying the "alternative patch" couldn't be enhanced to do these
things properly, but it would then no longer be a 20-line patch.  It
would also likely be much slower.  Why bother, when comment-cache exists
and works?

-- 
Alan Mackenzie (Nuremberg, Germany).