Re: Command substition failure?

[Michael Witten]

On Sat, 02 Apr 2011 15:20:23 +0200, Andreas Schwab wrote:

>> Could somebody please tell me what's going on here?
>
> Brace expansion.
>
> $ set -x
> $ echo "$(echo '"' | awk '{sub(/a/,$0)}')"
> ++ echo '"'
> ++ awk 'sub(/a/'
> awk: cmd. line:1: sub(/a/
> awk: cmd. line:1:        ^ unexpected newline or end of string
> ++ echo '"'
> ++ awk '$0)'
> awk: $0)
> awk:   ^ syntax error
> + echo '' ''

So, basically, bash is viewing this:

  echo "$(echo '"' | awk '{sub(/a/,$0)}')"

as:

  echo PREAMBLE{x,y}POSTSCRIPT

where PREAMBLE is the string:

  "$(echo '"' | awk '

and the `x' is the string:

  sub(/a/

and the `y' is the string:

  $0)

and the POSTSCRIPT is the string:

  ')"

Thus, bash expands the line to the following:

  echo "$(echo '"' | awk 'sub(/a/')" "$(echo '"' | awk '$0)')"

Bash runs the first substitution command:

  echo '"' | awk 'sub(/a/'

which fails with:

  awk: cmd. line:1: sub(/a/
  awk: cmd. line:1:        ^ unexpected newline or end of string

and produces an empty string as output.

Bash runs the second substitution command:

  echo '"' | awk '$0)'

which fails with:

  awk: $0)
  awk:   ^ syntax error

and produces an empty string as output.

Thus, the whole line becomes:

  echo "" ""

which outputs 2 characters: A space followed by a newline.

Phew!

OK, now, I think that's a bug!

The bash manual here:

  info '(bash)Brace Expansion'

or here:

  
http://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html#Brace-Expansion

says:

  3.5.1 Brace Expansion
  ---------------------

  ...
      A sequence expression takes the form `{X..Y[..INCR]}', where X and Y
   are either integers or single characters, and INCR, an optional
   increment, is an integer...

     Brace expansion is performed before any other expansions,
  and any characters special to other expansions are preserved
  in the result. It is strictly textual. Bash does not apply any
  syntactic interpretation to the context of the expansion or
  the text between the braces. To avoid conflicts with parameter
  expansion, the string `${' is not considered eligible for brace
  expansion.

     A correctly-formed brace expansion must contain unquoted
  opening and closing braces, and at least one unquoted comma
  or a valid sequence expression. Any incorrectly formed brace
  expansion is left unchanged.

     A { or `,' may be quoted with a backslash to prevent its
  being considered part of a brace expression. To avoid conflicts
  with parameter expansion, the string `${' is not considered
  eligible for brace expansion.
  ...

In particular:

     A correctly-formed brace expansion must contain unquoted
  opening and closing braces, and at least one unquoted comma
  or a valid sequence expression. Any incorrectly formed brace
  expansion is left unchanged.

Now, my email with `Message-ID':

  ecdf0813-3e67-4bb7-bfaf-6bfc12336c31-mfwitten@gmail.com

shows:

  The input characters within the quoted string
  that are also enclosed between "$(" and the
  matching ')' shall not be affected by the
  double-quotes, but rather shall define that
  command whose output replaces the "$(...)" when
  the word is expanded.

and this:

  ... When using the `$(COMMAND)' form, all characters between
  the parentheses make up the command; none are treated
  specially.

So, there should be no brace expansion in this case, because
the entire brace construct is quoted:

  '{sub(/a/,$0)}'

Thus, bash has a bug.

My guess is the nature of the problem is that the combination
of the outer-most quotes (which would render most characters
as literal) and the command substitution (which in some sense
is probably parsed in a `top-level' context) works to confuse
the brace-expansion logic.

Sincerely,
Michael Witteng