bug#54227: 29.0.50; [PATCH] Inconsistencies with Eshell variable interpolation

[Jim Porter]

There are a few related issues with how Eshell parses variable 
interpolations, especially when mixing with double-quotes. Starting from 
"emacs -Q --eval '(eshell)':

1. Quoted subcommands

  ~ $ echo ${echo hi}
  hi  ;; good, this is what we want
  ~ $ echo "${echo hi}"
  echo hi: command not found

In this case, Eshell gets confused by the double quotes and looks for 
the command named `echo hi', not the command named `echo' with the 
argument `hi'.

2. Complex list indexing

  ~ $ echo $PATH[: 0]
  /usr/local/bin  ;; good (the first element of your $PATH)
  ~ $ echo $PATH[":" 0]
  Wrong type argument: number-or-marker-p, (eshell-escape-arg ":")

Here, Eshell neglects to evaluate the subscript arguments, so only 
literal, unquoted values work. Note that the ":" in this case means 
"split the string $PATH with colon as a delimiter". The same happens 
with Elisp subexpressions:

  ~ $ echo $exec-path[0]
  /usr/local/bin  ;; good, again
  ~ $ echo $exec-path[$(+ 1 -1)]
  Wrong type argument: number-or-marker-p, (eshell-escape-arg (let 
((indices 'nil)) (eshell-command-to-value (eshell-lisp-command '(+ 1 -1)))))

3. Bare-string indexing

  ~ $ setq foo "fooXbarXbaz"
  fooXbarXbaz
  ~ $ echo $foo[X 0]
  (nil
    #("fooXbarXbaz" 0 11
    (escaped t)))

This is similar to the above case, but here Eshell thinks that "X" is a 
literal index to use. First, it splits `foo' into a list using space as 
a delimiter. Then gets confused when it uses a string as an index into a 
list, so it returns nil. The second element of the result is the 0th 
element of `foo'-as-a-list, aka the whole string since there are no 
spaces. (Just ignore for now the fact that the string "fooXbarXbaz" 
mysteriously gained an `escaped' property. That's a separate bug.)

This last one is less of a bug and more of a missing feature: if a bare 
(non-numeric) string is used as an index for a string variable, we know 
it *must* be a delimiter argument. Since we're indexing into a list, a 
string index doesn't really make sense.

--------------------

Attached is a patch series to fix these cases, as well as a lot more 
tests for Eshell variable interpolation, since this can get tricky. I've 
also updated the documentation to clarify how indexing works. I didn't 
initially realize that you could do quite so many complex things with 
it, so I figured it'd be good to explain in more detail so it's easier 
to discover.

0001-Move-Eshell-variable-interpolation-tests-to-their-ow.patch
Description: Text document

0002-Add-a-new-macro-to-simplify-parsing-temporary-Eshell.patch
Description: Text document

0003-Fix-Eshell-dollar-interpolation-inside-of-double-quo.patch
Description: Text document

0004-Fix-parsing-of-indices-in-Eshell-expansions.patch
Description: Text document

0005-Allow-splitting-strings-in-Eshell-expansions-with-pl.patch
Description: Text document