bug#43558: [PATCH]: Fix (forward-comment 1) when end delimiter is escaped.

[Alan Mackenzie]

Hello, Stefan.

On Thu, Sep 24, 2020 at 12:56:42 -0400, Stefan Monnier wrote:
> > As already said, this is a(n ugly) workaround.  syntax.c should handle
> > comments in all their generality.  With a bit of consideration, the
> > method to do this is clear:

> In my world, it's quite normal for a specific language's lexical rules
> not to line up 100% with syntax tables (whether for strings, comments,
> younameit).  I don't see anything very special here.

Normally when there's a mismatch, it's because a character is
syntactically ambiguous.  There's nothing syntax.c can do about this.

In the current situation, this isn't the case: syntax.c is unable to
handle a comment scenario where there is no ambiguity.

> A `syntax-propertize` rule for "\*/" should be very easy to implement
> and fairly cheap since the regexp is simple and will almost never match.

Well, the rule would actually be for escaped newlines, but this would be
quite expensive (compared with a syntax.c solution) since every comment
near a change region would need scanning at each change.

> So, yeah, you can add yet-another-hack on top of the other syntax.c
> hacks if you want, but there's a good chance it will only ever be used
> by CC-mode.  It will take a lot more code changes in syntax.c than
> a quick tweak to your Elisp code to search for "\*/".

I've hacked up a working, but as yet unsatisfactory, change to syntax.c.
It is surely better, where possible, to fix bugs at their point of
causation rather than by workarounds elsewhere.  As you note, CC Mode
modes will be the only known users at the moment.

Just as an aside, the project where I was working ~four years ago banned
a proprietory editor after a mammoth search for a bug caused by an
unintentional escaped NL on a line comment.  The banned editor didn't
fontify the continuation line in comment face.  I was able to
demonstrate to the project manager that Emacs fontified that comment
correctly.

> I do think it would be good to handle this without `syntax-table`
> text-property hacks, but I think that should come with an overhaul of
> syntax.c based on a major-mode provided DFA (or something like that) so
> it can accommodate all the various oddball cases without even the need
> to introduce the notion of escaping comment markers.

That sounds almost more like a rewrite than an overhaul.  You mean, I
think, that the syntax of language expressions would be defined using
something a bit like (but more powerful than) regular expressions.  And
with that, the need for syntactic analysis in Lisp would be much
reduced.

We would need to make sure that this wouldn't run more slowly than the
current syntax.c/Lisp combination.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).