Re: Semantics of font-lock-beginning-of-syntax-function

[Stefan Monnier]

> The docstring for font-lock-beginning-of-syntax-function says:
> 
>     Non-nil means use this function to move back outside of a
>     syntactic block.  When called with no args it should leave point
>     at the beginning of any enclosing syntactic block.
> 
> I'd like to know what - exactly - "syntactic block" means here. How is
> this function used, and what assumptions does the code that uses it
> have on the target position? A couple of guesses I've made more or
> less out of the blue are:
> 
> o  The point should be outside any comment or string literal.
> o  The point should be no later than the beginning of the current
>    line.

That looks right, although I think it's usually (always?) called from
the beginning of a line, so the second constraint is just "no later than
point".

> The word "block" suggests that it also should be at the beginning of a
> programmatic block, i.e. a paren sexp in C and similar languages. If
> so, why does font-lock need that? Afaics it doesn't operate on the
> structural level at all. Not knowing all the details, I thought it'd
> be enough to go to any preceding position that's outside comments and
> string literals.

Note that the details have slightly changed in Emacs-21.4 when I added
syntax-ppss and made font-lock use it.  `font-lock' only cares about
comments and strings, but syntax-ppss returns the same info as
`parse-partial-sexp' and might be used by any other code.
Also font-lock-beginning-of-syntax-function might be ignored if syntax-ppss
feels like the result is wrong (e.g. if the `face' property says
`font-lock-comment-face') or if it expects that its own cache of parse data
will be quicker.  This last optimization was also present in font-lock
before, although it doesn't work exactly in the same way (the current
caching in syntax-ppss is much more aggressive).

> If I interpret the second sentence correctly, it should move to (at
> least) the beginning of the outermost enclosing block. When and why is
> it necessary to move that far? Many times (e.g. in Java code) that's
> almost as bad as the beginning of the buffer since there might very
> well be a single class surrounding all the code in a file.

`syntax-ppss' currently assumes that `syntax-begin-function' (which
defaults to the value of `font-lock-beginning-of-syntax-function')
returns a position POS such that (parse-partial-sexp (point-min) POS)
is equivalent to nil (i.e. such that
(equal (pps POS (+ POS N)) (pps (point-min) (+ POS N)) modulo minor
discrepancies.

It's OK for `syntax-begin-function' (font-lock-beginning-of-syntax-function
should be obsoleted) not to move to the very top-level as long as users
of syntax-ppss don't depend on it.

Note also that the caching done in syntax-ppss makes `syntax-begin-function'
much less important.


        Stefan