Re: "Raw" string literals for elisp

[Anna Glasgall]

On Tue, 2021-09-07 at 21:49 -0400, Anna Glasgall wrote:
> 
[snip]
> I've successfully taught the elisp reader (read1 in lread.c) how to
> read r-strings. I thought I had managed to make lisp-mode/elisp-mode
> happy by allowing "r" to be a prefix character (C-x C-e and the
> underlying forward-sexp/backward-sexp seemed to work fine at first),
> but realized that I ran into trouble with strings containing the
> sequence of characters '\\"'.
> 
> The reader correctly reads r"a\\"" as a string containing the
> sequence
> of characters 'a', '\', '"', and M-: works. Unfortunately, if I try
> sexp-based navigation or e.g. C-x C-e, it falls apart. The parser in
> syntax.c, which afaict is what lisp-mode is using to try and find
> sexps
> in buffer text, doesn't seem to know what to do with this expression.
> I've spent some time staring at syntax.c, but I must confess that I'm
> entirely defeated in terms of what changes need to be made here to
> teach this other parser about prefixed strings in where the prefix
> has
> meaning that affects the interpretation of the characters between
> string fences.
> 

I've taken the suggestion Lars Ingebrigtsen made further up-thread and
implemented another draft of this using #r"" instead of r""; not only
does this not break existing elisp syntax (as noted by Po Lu earlier),
forward-sexp and backwards-sexp understand the new syntax without
having to be taught about it, to my considerable surprise. Or at least
they understand it to the same extent that they understand #s() or
#^[], anyway.

Alan (Dr. Mackenzie? Forgive me, not sure what standards are here),
your point about strings ending in \ is very well taken and I'm frankly
not sure what the easiest path forward here is. Having "raw literals
cannot end in a \" is a weird and unpleasant restriction, although the
fact that it is one that Python places on r-strings (to my considerable
surprise; I've been using Python since the mid-00s and have never run
across this particular syntax oddity before) may mean that it is
perhaps not so bad. The C++ concept of allowing r-strings to specify
their own delimiters is perhaps maximally flexible, but is definitely
going to be a heavier lift to implement than any of the above. I'd love
to hear people's opinions on the merits of the various possible
approaches here.

You've all given me a great deal of food for thought, which I will
attempt to digest before spinning a new revision of this patch.

thanks,

Anna

> I've attached a copy of my WIP patch; it's definitely not near final
> code quality and doesn't have documentation yet, all of which I would
> take care of before submitting for inclusion. I also haven't filled
> out
> the copyright assignment paperwork yet, but should this work reach a
> point where it was likely to be accepted, I'd be happy to do that.
> 
> 
> I'd very much appreciate some pointers on what to try next here, or
> some explanation of how syntax.c/syntax.el works beyond what's in the
> reference manual. If this is a fool's errand I'm tilting at here, I'd
> also appreciate being told that before I sink more time into it :)
> 
> thanks,
> 
> Anna Glasgall
> 
>