Re: [Fixed] NL character removed after \\ in command substitution

[Haojun Bao]

Here's the patch which I think fixes the problem:

diff --git a/parse.y b/parse.y
index df1231da..aca8ab62 100644
--- a/parse.y
+++ b/parse.y
@@ -2616,7 +2616,10 @@ pop_alias:
     }
 #endif /* ALIAS || DPAREN_ARITHMETIC */

-  if MBTEST(uc == '\\' && remove_quoted_newline &&
shell_input_line[shell_input_line_index] == '\n')
+    if MBTEST(uc == '\\' &&
+            unquoted_backslash == 1 &&
+            remove_quoted_newline &&
+            shell_input_line[shell_input_line_index] == '\n')
     {
  if (SHOULD_PROMPT ())
    prompt_again ();



On Tue, Aug 17, 2021 at 4:28 PM Haojun Bao <baohaojun@gmail.com> wrote:
>
> Thank you for the reply.
>
> Yes, we can add quotes to EOF, but in these simple reproducing steps,
> there are no variables.
>
> In my real use case, there will be variables, like the following,
> which makes adding quotes to <<EOF infeasible:
>
> cat <<"EOF"
> hello $HOME \\
> world
> EOF
>
> My point is, there seems to be some hole in the POSIX standard with
> regard to this behavior? Compare the same command using different
> shells:
>
> for s in bash zsh dash ksh mksh; do
>     echo -n "output from $s: "
>     $s -c 'echo "$(cat <<EOF
> hello \\
> world
> EOF
> )"
> ';
> done
>
> Output is:
>
> output from bash: hello \world
> output from zsh: hello \
> world
> output from dash: hello \
> world
> output from ksh: hello \
> world
> output from mksh: hello \
> world
>
> So, the bash output seems wrong to me, it's not POSIX compliant.
>
> On Tue, Aug 17, 2021 at 2:32 PM Andreas Kusalananda Kähäri
> <andreas.kahari@abc.se> wrote:
> >
> > On Tue, Aug 17, 2021 at 10:28:16AM +0800, Haojun Bao wrote:
> > > Configuration Information [Automatically generated, do not change]:
> > > Machine: x86_64
> > > OS: linux-gnu
> > > Compiler: gcc
> > > Compilation CFLAGS: -g -O2
> > > -fdebug-prefix-map=/build/bash-2bxm7h/bash-5.0=.
> > > -fstack-protector-strong -Wformat -Werror=format-security -Wall
> > > -Wno-parentheses -Wno-format-security
> > > uname output: Linux bhj-pc1 5.10.0-0.bpo.5-amd64 #1 SMP Debian
> > > 5.10.24-1~bpo10+1 (2021-03-29) x86_64 GNU/Linux
> > > Machine Type: x86_64-pc-linux-gnu
> > >
> > > Bash Version: 5.0
> > > Patch Level: 3
> > > Release Status: release
> > >
> > > Description:
> > >
> > > A bug found in parse.y, that will treat reading of COMMAND and
> > > $(COMMAND) differently, despite the info manual saying that:
> > >
> > > > When the old-style backquote form of substitution is used, backslash
> > > > retains its literal meaning except when followed by '$', '`', or '\'.
> > > > The first backquote not preceded by a backslash terminates the command
> > > > substitution.  When using the '$(COMMAND)' form, all characters between
> > > > the parentheses make up the command; none are treated specially.
> > >
> > > It seems the NL after \\ will be removed when run as $(COMMAND) in 
> > > parse.y?
> > >
> > > Repeat-By:
> > >
> > > This command will output $'hello \\\nworld\n':
> > >
> > > cat <<EOF
> > > hello \\
> > > world
> > > EOF
> > >
> > > This command will output $'hello \\world\n' (missing the \n after \\):
> > > echo "$(
> > > cat <<EOF
> > > hello \\
> > > world
> > > EOF
> > > )"
> >
> >
> > Consider quoting the here-document:
> >
> >         cat <<'EOF'
> >         hello \\
> >         world
> >         EOF
> >         )"
> >
> >
> > --
> > Andreas (Kusalananda) Kähäri
> > SciLifeLab, NBIS, ICM
> > Uppsala University, Sweden
> >
> > .