shell-expand-line drops quotation marks [FIXED]

[Dabrien 'Dabe' Murphy]

[NOTE]  Below is a message I started to write listing a whole slew of cases where `shell-expand-line` didn't Do What I Mean.

After familiarizing myself with the source code, however, I was pleasantly surprised to discover that there's actually a one line one character fix for almost every case I came up with(!)

So, without further ado...

  ----  8<  ----  8<  ----  Cut Here  ----  8<  ----  8<  ----

I know this thread is a year old, but I do have to say I agree with the OP that `shell-expand-line`'s decision to perform Quote Removal seems to violate the Principle of Least Astonishment...

To say `shell-expand-line` "Expand[s] the line as the shell does" seems, shall we say, "disingenuous" — if not an outright lie...  The shell preserves whitespace:

    ### Helper Function ###
    prompt% argDump() { while [[ $# -gt 0 ]]; do echo "ARG: '$1'"; shift; done; }

    prompt% argDump "one two three"
    ARG: 'one two three'

whereas `shell-expand-line` does not.

    prompt% argDump "one two three" [<M-C-e>]
    prompt% argDump one two three       # FAIL FIXED
    ARG: 'one'
    ARG: 'two'
    ARG: 'three'


Quote Removal makes sense during command EXECUTION (since you wouldn't want your quotes passed in with the arguments) but it doesn't make sense during (readline) EDITING, IMHO...

 
Consider the following variable expansion:

    prompt% foo="one two"
    prompt% argDump $foo
    ARG: 'one'
    ARG: 'two'

    prompt% argDump "$foo"
    ARG: 'one two'

    prompt% argDump $foo [<M-C-e>]
    prompt% argDump one two    # So far, so good, actually...

    prompt% argDump "$foo" [<M-C-e>]
    prompt% argDump one two    # FAIL FIXED

Conversely:  [This is one case I have yet to solve; I imagine it involves a carefully placed `sh_backslash_quote_for_double_quotes()`, but I haven't worked it out, yet...]

    prompt% foo='"one two"'    # Double quotes inside single quotes
    prompt% echo $foo
    "one two"

    prompt% argDump $foo
    ARG: '"one'
    ARG: 'two"'                # I'm happy with that...
    prompt% argDump $foo [<M-C-e>]
    prompt% argDump "one two"
    ARG: 'one two'             # ARGH!!


Command Injection, anyone?

    prompt% bar='; date'
    prompt% echo $bar [<M-C-e> <Enter>]
    Wed Nov  2 15:28:29 EDT 2016            # Fool me once...

    prompt% echo "$bar" [<M-C-e> <Enter>]   # Better Use Protection!
    Wed Nov  2 15:28:41 EDT 2016            # FAIL FIXED


At the very least, I would expect `shell-expand-line` to be more or less idempotent; expanding the line multiple times shouldn't change the behavior of the command that actually gets executed:

    prompt% echo $'\007'
    # Generates ^G (BEL)

    prompt% echo $'\007' [<M-C-e> <M-C-e> <M-C-e>]
    # Becomes...
    prompt% echo $\007
    prompt% echo $007
    prompt% echo -bash07  # WTF??! How'd $0 get in there? FIXED


Ditto history expansion:

    prompt% echo $'\007'
    prompt% echo !$ [<M-C-e>]
    prompt% echo $\007        # FAIL FIXED
    $007


I understand it's hard to do the Right Thing sometimes:  [Still unsolved]

    prompt% alias ls="ls -F"
    prompt% ls [<M-C-e>]
    prompt% ls -F [<M-C-e>]
    prompt% ls -F -F [<M-C-e>]
    prompt% ls -F -F -F
    ...


I thought that maybe prepending "command" or "\" might help, but you'd still have:

    prompt% alias ls="ls -F"
    prompt% alias qq="ls"
    prompt% qq /bin/bash
    /bin/bash*                # with '*', good

    prompt% alias qq="\ls"    # Note the backslash
    prompt% qq /bin/bash
    /bin/bash                 # zero '*', also good

    prompt% qq /bin/bash [<M-C-e>]
    prompt% ls /bin/bash
    /bin/bash*                # WITH '*'... BAD :-(  FIXED

(It's worth noting that the shell itself is smart enough to figure out when to stop expanding aliases; it seems like that logic would be useful here, as well...)


And lastly, even when you ARE doing something "correctly", you *STILL* have to watch out for pitfalls:

    prompt% alias su='sudo -sEp '\''[sudo] password for %p: '\'''

    prompt% su
    [sudo] password for dabe: ********

    prompt% su [<M-C-e>]
    prompt% sudo -sEp [sudo] password for %p:    # FAIL FIXED
    [sudo]********
    /bin/bash: password: command not found


So what's the fix, you might ask?

diff --git a/bashline.c b/bashline.c
index 238a190..e17a49d 100644
--- a/bashline.c
+++ b/bashline.c
@@ -2689,7 +2689,7 @@ shell_expand_line (count, ignore)
       /* If there is variable expansion to perform, do that as a separate
         operation to be undone. */
       new_line = savestring (rl_line_buffer);
-      expanded_string = expand_string (new_line, 0);
+      expanded_string = expand_string (new_line, 1);
       FREE (new_line);
       if (expanded_string == 0)
        {

If you're really concerned that people are actually relying on the old behavior, I'm sure it would be easy to create some sort of "shell-expand-preserve-quotes" readline variable, or some such...  Show me where to submit a Pull Request and I'd be happy to whip one up!  :-D


PS — Another example where `shell-expand-line` decidedly does NOT "expand the line as the shell does" is with globs and tilde prefixes, but I'm aware this is a known limitation:

    prompt% echo ~/.bash* [<M-C-e>]

Cheers,

--
:- Dabe