bug#49518: 28.0.50; electric-pair-mode skip-self fails for single-quotes in python-mode

[João Távora]

[Stefan, I'm CC-ing you to assist with a python.el syntax propertization
bug and a patch for it at the end of this mail]

Jim Porter <jporterbugs@gmail.com> writes:

> On 9/18/2021 5:20 PM, João Távora wrote:
>> Here is the patch.  Give it a shot.
>> diff --git a/lisp/elec-pair.el b/lisp/elec-pair.el
>> index d8c377a2ef..b8b8a97651 100644
>> --- a/lisp/elec-pair.el
>> +++ b/lisp/elec-pair.el
>> @@ -198,7 +198,9 @@ electric-pair-syntax-info
>>   inside a comment or string."
>>     (let* ((pre-string-or-comment (or (bobp)
>>                                       (nth 8 (save-excursion
>> -                                             (syntax-ppss (1- (point)))))))
>> +                                             (skip-chars-backward
>> +                                              (string command-event))
>> +                                             (syntax-ppss (point))))))
>>            (post-string-or-comment (nth 8 (syntax-ppss (point))))
>>            (string-or-comment (and post-string-or-comment
>>                                    pre-string-or-comment))
>> 
>
> Thanks, this patch works for me. 
>> Maybe better less brittle solutions can probably be found within the
>> syntax tables framework.
> I agree that something less brittle would be nice, but that might be
> tricky. I had tried a few options when I filed this bug, but nothing
> worked quite right.

The point of 'electric-pair-syntax-info's STRING-OR-COMMENT-START return
value is to tell if both the current and the pre-insertion point are and
were inside a string or a comment.  It seems reasonable (i.e. not crazy)
to skip the same syntax _and_ char backwards to discover that.  Ideally
this would be remembered from before the insertion, it's true.  But
again, the fact that the patch passes all demanding tests is a good
sign: they are quite strict.  Notice, for example, that if the patch
were the seemingly also reasonable:

       (let* ((pre-string-or-comment (or (bobp)
                                         (nth 8 (save-excursion
    -                                             (skip-chars-backward
    -                                              (string command-event))
    +                                             (skip-syntax-backward
    +                                              (string (char-syntax 
command-event)))
                                                  (syntax-ppss (point))))))
              (post-string-or-comment (nth 8 (syntax-ppss (point))))
              (string-or-comment (and post-string-or-comment

There would be one single test failure in a ruby-mode "mixed quote"
situation.

> As for Stephen's desired behavior, the code snippet I posted above[1]
> mostly fixes it (just the `python-electric-pair-string-delimiter' part
> is necessary after your patch). However, that code doesn't work right
> if `electric-pair-skip-self' is set to nil, so I'd need to be a bit
> smarter with the implementation if I wanted to merge it into
> Emacs. Perhaps there's an even cleaner solution though.

I think the cleaner solution is to fix a bug in python.el.  Here's the
reproduction for that bug (no electric pair mode involved)

    emacs -Q something.py
    type two single quotes
    M-: (nth 3 (syntax-ppss))
    notice how the return value says you're outside a string, correctly
    type another quote
    M-: (nth 3 (syntax-ppss))
    notice how the return value says you're inside a string, correctly
    backspace the quote just entered
    M-: (nth 3 (syntax-ppss))
    notice how the return value says you're inside a string, incorrectly

When this bug is fixed, I'm confident that Stephen's case will start
working.

In fact, it's possible that if this python.el bug were fixed, we
wouldn't need the above patch at all, becasue backtracking one char into
just before the third quote of a triple quote should _also_ yield a nil
(nth 3 (syntax-ppss)).

In fact, I have tried a patch for python.el that could fix _all_ of this
(but would need some heavy testing probably):
 
     (defvar python-mode-syntax-tablediff --git a/lisp/progmodes/python.el 
b/lisp/progmodes/python.el
    index 19b79b6919..da7b92ae42 100644
    --- a/lisp/progmodes/python.el
    +++ b/lisp/progmodes/python.el
    @@ -775,12 +775,14 @@ python-syntax-stringify
                ;; The first quote is escaped, so it's not part of a triple 
quote!
                (goto-char (1+ quote-starting-pos)))
               ((null string-start)
    -           ;; This set of quotes delimit the start of a string.
    -           (put-text-property quote-starting-pos (1+ quote-starting-pos)
    +           ;; This set of quotes delimit the start of a string.  Put
    +           ;; the delimiter syntax in the last of the three quotes.
    +           (put-text-property (1- quote-ending-pos) quote-ending-pos
                                   'syntax-table (string-to-syntax "|")))
               (t
    -           ;; This set of quotes delimit the end of a string.
    -           (put-text-property (1- quote-ending-pos) quote-ending-pos
    +           ;; This set of quotes delimit the end of a string.  Put the
    +           ;; delimiter syntax in the first of the three quotess.
    +           (put-text-property quote-starting-pos (1+ quote-starting-pos)
                                   'syntax-table (string-to-syntax "|"))))))
     
     (defvar python-mode-syntax-table

João