Re: weird bash5 bug with ``for i; do echo; done; echo in''

[Eduardo A . Bustamante López]

On Sat, Mar 24, 2018 at 06:05:07PM +0000, Dennis Williamson wrote:
[...]
> >> $ bash5 -c 'for i; do echo; done; echo in'
> >> bash5: -c: line 0: syntax error near unexpected token `in'
> >> bash5: -c: line 0: `for i; do echo; done; echo in'
[...]
> bash5 -c 'for i; do echo in; done'
> 
> fails in the same way.

A shorter reproducer:

  for i do : in; done

f698849a75fc781472806182c3dc930077a5d828 is the last known good commit that
behaves correctly. It seems like the changes committed in
124d67cde092330257c69e8c9bd64af40ffd9b73 
(http://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel&id=124d67cde092330257c69e8c9bd64af40ffd9b73)
introduced this problem:

|                                  3/25
|                                  ----
| parse.y
|       - read_token_word: if we read a WORD immediately after a CASE, FOR, or
|         SELECT, increment `expecting_in_token' to note that we're going to
|         be reading IN as the next non-newline token
|       - special_case_tokens: if we're expecting an IN (expecting_in_token > 0)
|         and we have an "in", return IN and decrement expecting_in_token. This
|         is a better implementation of Posix grammar rule 6 than the previous
|         code in that function (which is still there for posterity, but should
|         eventually be removed).  Fixes bug reported by Torka Noda
|         <kanito.tasoga@gmail.com>


The following is not a fix, but it does make the problem go away:

diff --git a/parse.y b/parse.y
index 29b59f81..9efcc4aa 100644
--- a/parse.y
+++ b/parse.y
@@ -5315,7 +5315,7 @@ got_token:
       if (word_top < MAX_CASE_NEST)
        word_top++;
       word_lineno[word_top] = line_number;
-      expecting_in_token++;
+      //expecting_in_token++;
       break;
     }
 

The problem is that expecting_in_token is increased unconditionally for a `for'
statement, but the `in WORD' part of the loop is optional, so this confuses the
parser.