Backslashed space in variable/argument expansion does not prevent splitting.

[Dean Wakerley]

From: deanw
To: bug-bash@gnu.org
Subject: Backslashed space in variable expansion does not prevent splitting.

Configuration Information [Automatically generated, do not change]:
Machine: i686
OS: cygwin
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash.exe' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='cygwin' -DCONF_MACHTYPE='i686-pc-cygwin' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -DRECYCLES_PIDS   -I.  -I/usr/src/bash-4.3.42-4.i686/src/bash-4.3 -I/usr/src/bash-4.3.42-4.i686/src/bash-4.3/include -I/usr/src/bash-4.3.42-4.i686/src/bash-4.3/lib  -DWORDEXP_OPTION -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/usr/src/bash-4.3.42-4.i686/build=/usr/src/debug/bash-4.3.42-4 -fdebug-prefix-map=/usr/src/bash-4.3.42-4.i686/src/bash-4.3=/usr/src/debug/bash-4.3.42-4
uname output: CYGWIN_NT-6.1-WOW DW03PC02 2.5.1(0.297/5/3) 2016-04-21 22:12 i686 Cygwin
Machine Type: i686-pc-cygwin

Bash Version: 4.3
Patch Level: 42
Release Status: release

Description:
 The expansion of a variable containing spaces splits on all spaces including escaped/backslashed space.
   
    I ran into this problem when trying to buiild a TCL extension. Building the extension requires pulling in
    with source tclConfig.sh which provides various settings. TCL_DEFS contains space separated -D arguments
    to pass to gcc. Some of these arguements contains space which are backslashed to prevent splitting.
   
    Argument expansion is not distinguishing between space and backslased space.
    I have tried 3.00.16(1)-release on FedoraCore 4, 4.2.10(1)-release on FedoraCore 16, 4.3.42(4)-release on Cygwin 2.5.1
    with the same outcome.

Repeat-By:
        Run the ./bad_exp.sh script. It explains the desired behaviour and demonstrates issue. I include below the contents of two script and my output log.

=====Start File ./bad_exp.sh
#!/usr/bin/env bash
# Issue: Backslash space is not preventing argument splitting

echo $BASH_VERSION

# TCL_DEFS is actually source-d from TCL config script so have no control over it.
TCL_DEFS='-DPACKAGE_NAME=\"tcl\" -DPACKAGE_TARNAME=\"tcl\" -DPACKAGE_VERSION=\"8.5\" -DPACKAGE_STRING=\"tcl\ 8.5\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_LIMITS_H=1 -DHAVE_SYS_PARAM_H=1 -DTCL_CFGVAL_ENCODING=\"iso8859-1\" -DMODULE_SCOPE=extern\ __attribute__\(\(__visibility__\(\"hidden\"\)\)\) -DTCL_CFG_DO64BIT=1 -DHAVE_CAST_TO_UNION=1 -DTCL_SHLIB_EXT=\".so\" -DNDEBUG=1 -DTCL_CFG_OPTIMIZED=1 -DTCL_TOMMATH=1 -DMP_PREC=4 -D_LARGEFILE64_SOURCE=1 -DTCL_WIDE_INT_IS_LONG=1 -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRTOL=1 -DHAVE_WAITPID=1 -DHAVE_GETADDRINFO=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_GMTIME_R=1 -DHAVE_LOCALTIME_R=1 -DHAVE_MKTIME=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DHAVE_STRUCT_STAT_ST_BLOCKS=1 -DHAVE_STRUCT_STAT_ST_BLKSIZE=1 -DHAVE_BLKCNT_T=1 -DHAVE_INTPTR_T=1 -DHAVE_UINTPTR_T=1 -DHAVE_SIGNED_CHAR=1 -DHAVE_LANGINFO=1 -DHAVE_SYS_IOCTL_H=1 -DTCL_UNLOAD_DLLS=1 -DHAVE_CPUID=1 '

./echo_args.sh ${TCL_DEFS}

echo 'Reduction of problem'
echo 'The desired result is echo_args should receive three arguments arg0, arg1 and "arg 2"'
T='arg0 arg1 arg\ 2'

echo 'This receives four arguments, wrongly splitting on the backslashed space'
./echo_args.sh $T

echo 'This correctly receives one argument but is not the desired result'
./echo_args.sh "$T"
=====End File ./bad_exp.sh
=====Start File echo_args.sh
#!/usr/bin/env bash
# Script to display command line arguments

echo "${#} arguments"
i=0
for arg in "${@}"
do
    echo "${i}:$arg"
    ((i++))
done
=====End File echo_args.sh
=====Start output log
4.2.10(1)-release
53 arguments
0:-DPACKAGE_NAME=\"tcl\"
1:-DPACKAGE_TARNAME=\"tcl\"
2:-DPACKAGE_VERSION=\"8.5\"
3:-DPACKAGE_STRING=\"tcl\
4:8.5\"
5:-DPACKAGE_BUGREPORT=\"\"
6:-DSTDC_HEADERS=1
7:-DHAVE_SYS_TYPES_H=1
8:-DHAVE_SYS_STAT_H=1
9:-DHAVE_STDLIB_H=1
10:-DHAVE_STRING_H=1
11:-DHAVE_MEMORY_H=1
12:-DHAVE_STRINGS_H=1
13:-DHAVE_INTTYPES_H=1
14:-DHAVE_STDINT_H=1
15:-DHAVE_UNISTD_H=1
16:-DHAVE_LIMITS_H=1
17:-DHAVE_SYS_PARAM_H=1
18:-DTCL_CFGVAL_ENCODING=\"iso8859-1\"
19:-DMODULE_SCOPE=extern\
20:__attribute__\(\(__visibility__\(\"hidden\"\)\)\)
21:-DTCL_CFG_DO64BIT=1
22:-DHAVE_CAST_TO_UNION=1
23:-DTCL_SHLIB_EXT=\".so\"
24:-DNDEBUG=1
25:-DTCL_CFG_OPTIMIZED=1
26:-DTCL_TOMMATH=1
27:-DMP_PREC=4
28:-D_LARGEFILE64_SOURCE=1
29:-DTCL_WIDE_INT_IS_LONG=1
30:-DHAVE_GETCWD=1
31:-DHAVE_OPENDIR=1
32:-DHAVE_STRTOL=1
33:-DHAVE_WAITPID=1
34:-DHAVE_GETADDRINFO=1
35:-DUSE_TERMIOS=1
36:-DHAVE_SYS_TIME_H=1
37:-DTIME_WITH_SYS_TIME=1
38:-DHAVE_GMTIME_R=1
39:-DHAVE_LOCALTIME_R=1
40:-DHAVE_MKTIME=1
41:-DHAVE_TM_GMTOFF=1
42:-DHAVE_TIMEZONE_VAR=1
43:-DHAVE_STRUCT_STAT_ST_BLOCKS=1
44:-DHAVE_STRUCT_STAT_ST_BLKSIZE=1
45:-DHAVE_BLKCNT_T=1
46:-DHAVE_INTPTR_T=1
47:-DHAVE_UINTPTR_T=1
48:-DHAVE_SIGNED_CHAR=1
49:-DHAVE_LANGINFO=1
50:-DHAVE_SYS_IOCTL_H=1
51:-DTCL_UNLOAD_DLLS=1
52:-DHAVE_CPUID=1
Reduction of problem
The desired result is echo_args should receive three arguments arg0, arg1 and "arg 2"
This receives four arguments, wrongly splitting on the backslashed space
4 arguments
0:arg0
1:arg1
2:arg\
3:2
This correctly receives one argument but is not the desired result
1 arguments
0:arg0 arg1 arg\ 2
=====End output log