bug#58780: python.el infinite loop in info-current-defun

[kobarity]

As JD Smith pointed out, this problem is caused by
`python-nav-end-of-statement' moving point backward.  A simple example
illustrating the problem of `python-nav-end-of-statement' is as
follows:

#+begin_src python
' """
a = 1
#+end_src

When point is placed on the second line and
`python-nav-end-of-statement' is executed, point will move to the end
of the first line.

Here is why this happens.

`python-nav-end-of-statement' moves point to the end of the second
line, and checks if it is in the middle of the string using:

(python-syntax-context 'string)

It tells that the point is within the string started at the
single-quote in the first line, and the variable `string-start' is set
to the start of the string (single-quote in the first line).  Then,
point is moved to the start of the string, and the end of the string
is searched for using:

(re-search-forward (rx (syntax string-delimiter)) nil t)

It finds the triple double-quotes and moves point immediately after
them.  The variable `last-string-end' is set to this position.

In the second iteration of the while loop, the variable `string-start'
is set to the single-quote in the first line again.  This results in
exiting the loop because the following condition is not met.

(>= string-start last-string-end)

There are several issues here:

1. Although `python-syntax-context' detects both string-quote
(single quote) and string-delimiter (triple quotes) as the start of
string, only string-delimiter is searched for as the end of string.
2. The triple double-quotes is marked as string-delimiter even though
they are within the string begins with the single-quote.
3. As the end of the first line is not escaped using a backslash, the
string begins with the single-quote should not be continued on the
second line.

I would like to discuss the second and third issues separately.  The
first issue is the main cause of Bug#58780.  This issue also causes
another type of problem, illustrated by the following example.

#+begin_src python
abc = 'a\
b\
c'
d = '''d'''
#+end_src

When `python-nav-end-of-statement' is executed with point located at
the second line, point is moved to the end of the fourth line instead
of the third line.  This is a wrong point move for a correct Python
program.

The attached
0001-Fix-searching-for-end-of-string-in-python-nav-end-of.patch is my
proposal to fix this issue with some ERTs.  `forward-sexp' (in fact
`python-nav--lisp-forward-sexp') is added to search for the end of the
string if the string begins with single single/double-quote (' or ").

I think this 0001 patch also fixes Bug#56271.  At least, it passes the
ERT python-nav-end-of-block-2 which is introduced in Bug#56271, even
the workaround introduced in Bug#56271 is reverted.  The attached
0002-Revert-workaround-introduced-in-Bug-56271.patch is a patch to
revert the workaround.  If the 0001 patch is accepted, the 0002 patch
can also be applied.  But if we want to be safer, we can leave the
workaround as it is by not applying the 0002 patch.

I look forward to your comments.

0001-Fix-searching-for-end-of-string-in-python-nav-end-of.patch
Description: Binary data

0002-Revert-workaround-introduced-in-Bug-56271.patch
Description: Binary data