Re: Debugging SMIE for sh-script.el

[Stefan Monnier]

> returned by the smie lexer as "" when it runs into them from the outside.

There's no such thing as "the smie lexer":
Every major mode uses the lexer it wants with the behavior it fancies.

If the lexer decides to return nil when bumping into a char with paren
or string syntax then SMIE will treat it as a sign that this should be
handled in the "normal" way.  The main benefit is that paired parens can
then be skipped with things like `forward-sexp` which is much faster
than letting SMIE do the lexing+parsing inside the pair.

But a lexer doesn't have to do that.  An example would be when the
parens are escaped, e.g, in ELisp, `foo\(toto\)bar` is a single symbol
so an SMIE lexer for ELisp should handle it as a single token.


        Stefan