bug#62142: 30.0.50; python-fill-paragraph problems on filling strings

[Eli Zaretskii]

> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  João Távora <joaotavora@gmail.com>
> Date: Sun, 12 Mar 2023 17:39:37 +0900
> From: kobarity <kobarity@gmail.com>
> 
> The cause of the first problem is similar to Bug#58780.  In
> `python-fill-string', the string is searched for with the following
> code:
> 
>            (or (python-syntax-context 'string)
>                (and (equal (string-to-syntax "|")
>                            (syntax-after (point)))
>                     (point)))))
> 
> and the end of the string is searched for with the following code:
> 
>             (or (re-search-forward (rx (syntax string-delimiter)) nil t)
>                 (goto-char (point-max)))
> 
> These codes used to be appropriate, but it is not correct after the
> following commit which is included in Emacs 27 and later:
> 
> commit 4b39b741f1949ebad1dfccc5032dfce521bedc2a
> Author: Stefan Monnier <monnier@iro.umontreal.ca>
> Date:   Tue Apr 9 15:08:21 2019 -0400
> 
>     python.el: don't syntax-propertize single/double quoted strings
> 
>     * lisp/progmodes/python.el (python-syntax-propertize-function):
>     Only mark triple-quoted strings, let the normal syntax-table handle
>     the rest.
>     (python-syntax-stringify): Adjust accordingly.
> 
> From this commit, single quoted strings can not be identified by
> string-delimiter and can only be identified by string-quote.
> 
> In the fix of Bug#58780, I added `forward-sexp' to search for the end
> of single quoted string.  However, considering the third problem, I
> would prefer to have `python-fill-paragraph' and `python-fill-string'
> search only for triple quoted strings in this case.
> 
> 
> The second problem occurs after the following commit included in Emacs
> 28 and later:
> 
> commit 0646c6817139aa905a2f6079fdc82eb4be944de0
> Author: João Távora <joaotavora@gmail.com>
> Date:   Sun Sep 19 11:42:20 2021 +0100
> 
>     Make syntax-ppss more accurate for Python triple quotes (bug#49518)
> 
>     By putting delimiter syntax on the last character of Python
>     triple-quoted strings, this makes syntax-ppss be more accurate.
> 
> This commit narrows the range of string-delimiter from outermost
> quotes to innermost quotes, and results in `python-fill-paragraph' not
> recognizing the string when point is at some quotes.
> 
> 
> Attached is a patch to fix these problems.  Main changes are:
> 
> 1. single-quoted-string and triple-quoted-string are added as a TYPE
>    argument of `python-syntax-context'.
> 2. New helper function `python-info-triple-quoted-string-p' is added
>    to detect triple quoted strings including quotes.
> 3. `python-fill-paragraph' and `python-fill-string' use
>    `python-info-triple-quoted-string-p' to detect triple quoted
>    strings only.
> 
> I look forward to your comments.

Stefan and João, any comments?  If not, I'd like to install this on
emacs-29 soon.