Re: NL character removed after \\ in command substitution

[Haojun Bao]

Thank you for the reply.

Yes, we can add quotes to EOF, but in these simple reproducing steps,
there are no variables.

In my real use case, there will be variables, like the following,
which makes adding quotes to <<EOF infeasible:

cat <<"EOF"
hello $HOME \\
world
EOF

My point is, there seems to be some hole in the POSIX standard with
regard to this behavior? Compare the same command using different
shells:

for s in bash zsh dash ksh mksh; do
    echo -n "output from $s: "
    $s -c 'echo "$(cat <<EOF
hello \\
world
EOF
)"
';
done

Output is:

output from bash: hello \world
output from zsh: hello \
world
output from dash: hello \
world
output from ksh: hello \
world
output from mksh: hello \
world

So, the bash output seems wrong to me, it's not POSIX compliant.

On Tue, Aug 17, 2021 at 2:32 PM Andreas Kusalananda Kähäri
<andreas.kahari@abc.se> wrote:
>
> On Tue, Aug 17, 2021 at 10:28:16AM +0800, Haojun Bao wrote:
> > Configuration Information [Automatically generated, do not change]:
> > Machine: x86_64
> > OS: linux-gnu
> > Compiler: gcc
> > Compilation CFLAGS: -g -O2
> > -fdebug-prefix-map=/build/bash-2bxm7h/bash-5.0=.
> > -fstack-protector-strong -Wformat -Werror=format-security -Wall
> > -Wno-parentheses -Wno-format-security
> > uname output: Linux bhj-pc1 5.10.0-0.bpo.5-amd64 #1 SMP Debian
> > 5.10.24-1~bpo10+1 (2021-03-29) x86_64 GNU/Linux
> > Machine Type: x86_64-pc-linux-gnu
> >
> > Bash Version: 5.0
> > Patch Level: 3
> > Release Status: release
> >
> > Description:
> >
> > A bug found in parse.y, that will treat reading of COMMAND and
> > $(COMMAND) differently, despite the info manual saying that:
> >
> > > When the old-style backquote form of substitution is used, backslash
> > > retains its literal meaning except when followed by '$', '`', or '\'.
> > > The first backquote not preceded by a backslash terminates the command
> > > substitution.  When using the '$(COMMAND)' form, all characters between
> > > the parentheses make up the command; none are treated specially.
> >
> > It seems the NL after \\ will be removed when run as $(COMMAND) in parse.y?
> >
> > Repeat-By:
> >
> > This command will output $'hello \\\nworld\n':
> >
> > cat <<EOF
> > hello \\
> > world
> > EOF
> >
> > This command will output $'hello \\world\n' (missing the \n after \\):
> > echo "$(
> > cat <<EOF
> > hello \\
> > world
> > EOF
> > )"
>
>
> Consider quoting the here-document:
>
>         cat <<'EOF'
>         hello \\
>         world
>         EOF
>         )"
>
>
> --
> Andreas (Kusalananda) Kähäri
> SciLifeLab, NBIS, ICM
> Uppsala University, Sweden
>
> .