IFS boundary behaviour is not consistent passing arguments to a function for bash 4.3 or 4.4 .

[McCue, Glenn (SSC/SPC)]

From: gmccue
To: bug-bash@gnu.org
Subject:  IFS boundary error - bash 4.3 or 4.4 function call

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.3 
-L/home/abuild/rpmbuild/BUILD/bash-4.3/../readline-6.3
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -fmessage-length=0 
-grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 
-funwind-tables -fasynchronous-unwind-tables -g  -D_GNU_SOURCE -DRECYCLES_PIDS 
-Wall -g -Wuninitialized -Wextra -Wno-unprototyped-calls -Wno-switch-enum 
-Wno-unused-variable -Wno-unused-parameter -Wno-parentheses -ftree-loop-linear 
-pipe -DBNC382214=0 -DIMPORT_FUNCTIONS_DEF=0 -fprofile-use
uname output: Linux MYSQL-PROD-05 4.4.59-92.24-default #1 SMP Thu Jun 22 
14:29:09 UTC 2017 (d11a83a) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu

Bash Version: 4.3
Patch Level: 42
Release Status: release

Description:
        When using IFS and passing arguments to a function are the arguments 
are not divided at the IFS boundary unless the arguments are first concatenated 
to a single variable. 

Repeat-By:

The original bash scripts were written using bash-3.2 using IFS and passing 
argument to a function behaved as expected.
This is the bash-3.2 Configuration Information.
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/usr/src/packages/BUILD/bash-3.2 
-L/usr/src/packages/BUILD/bash-3.2/../readline-5.2
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -fmessage-length=0 -O2 -Wall 
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables 
-fasynchronous-unwind-tables -g  -D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g 
-std=gnu89 -Wextra -Wno-unprototyped-calls -Wno-switch-enum 
-Wno-unused-variable -Wno-unused-parameter -ftree-loop-linear -pipe 
-DIMPORT_FUNCTIONS_DEF=0
uname output: Linux DBSSSC01 3.0.101-107-default #1 SMP Thu Jun 22 14:37:55 UTC 
2017 (414ea9f) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu
Bash Version: 3.2
Patch Level: 57
Release Status: release
-------------
But with 4.3

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.3 
-L/home/abuild/rpmbuild/BUILD/bash-4.3/../readline-6.3
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -fmessage-length=0 
-grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 
-funwind-tables -fasynchronous-unwind-tables -g  -D_GNU_SOURCE -DRECYCLES_PIDS 
-Wall -g -Wuninitialized -Wextra -Wno-unprototyped-calls -Wno-switch-enum 
-Wno-unused-variable -Wno-unused-parameter -Wno-parentheses -ftree-loop-linear 
-pipe -DBNC382214=0 -DIMPORT_FUNCTIONS_DEF=0 -fprofile-use
uname output: Linux MYSQL-PROD-05 4.4.59-92.24-default #1 SMP Thu Jun 22 
14:29:09 UTC 2017 (d11a83a) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu
Bash Version: 4.3
Patch Level: 42
Release Status: release
---------------
And bash 4.4

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' 
-DCONF_VENDOR='unknown' -DLOCALEDIR='/home/gmccue/bash/bash-4.4/share/locale' 
-DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -g 
-O2 -Wno-parentheses -Wno-format-security
uname output: Linux DBSSSC01 3.0.101-107-default #1 SMP Thu Jun 22 14:37:55 UTC 
2017 (414ea9f) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu
Bash Version: 4.4
Patch Level: 0
Release Status: release
----------------
The IFS behaves differently depending on how the command is structured.
This is my test script. It demonstrates the behavior change between 3.2  and 
4.[3|4].

The function "doit" lists the individual arguments that it has received.
 
The function "doit" is called twice,  each time setting  IFS. 

The first time the function is called the arguments to "doit" are just listed 
in the function call.

The second time the function "doit" is called the arguments are concatenated in 
a variable and the variable is used as the argument to the function.
 
Here is test code.
----------------------------------
#!/bin/bash

echo BASH_VERSION=$BASH_VERSION

# list arguments
function doit
{
        declare -i argv_count=1
        echo "doit argc=$#"
        argv1=$1
        shift 1
        echo agv1=[$argv1]
        while [ $# -gt 0 ]
        do
                argv_count+=1;
                echo "argv_count=$argv_count[$1]"
                shift 1
        done
}

first_arg="1 "

## list arguments in the function call.
echo "first_arg=$first_arg"
echo "RESULT_A=\`doit \"\$first_arg\":2:3:4:\$PATH\'"
OIFS=$IFS
IFS=:
RESULT_A=`doit $first_arg:2:3:4:$PATH`
status=$?
IFS=$OIFS
echo -e "\$RESULT_A=\n$RESULT_A"
echo

## list arguments in a variable and use that in the function call.
all_args="$first_arg:2:3:4:$PATH"
echo "RESULT_B=\`doit \$all_args\'"
OIFS=$IFS
IFS=:
RESULT_B=`doit $all_args`
status=$?
IFS=$OIFS
echo -e "\$RESULT_B=\n$RESULT_B"
echo

# end of file
----------------------------------------
TEST RESULTS

Using bash3.2 both function calls act as expected. The arguments are divided 
for the function  at the IFS boundary.

BASH_VERSION=3.2.57(2)-release
first_arg=1 
RESULT_A=`doit "$first_arg":2:3:4:$PATH\'
$RESULT_A=
doit argc=14
agv1=[1 ]
argv_count=2[2]
argv_count=3[3]
argv_count=4[4]
argv_count=5[/usr/local/bin]
argv_count=6[/usr/bin]
argv_count=7[/bin]
argv_count=8[/usr/bin/X11]
argv_count=9[/usr/X11R6/bin]
argv_count=10[/usr/games]
argv_count=11[/opt/bin]
argv_count=12[/usr/lib/mit/bin]
argv_count=13[/usr/lib/mit/sbin]
argv_count=14[/u01/mysql/share/bin]

all_args="$first_arg:2:3:4:$PATH"
RESULT_B=`doit $all_args`
$RESULT_B=
doit argc=14
agv1=[1 ]
argv_count=2[2]
argv_count=3[3]
argv_count=4[4]
argv_count=5[/usr/local/bin]
argv_count=6[/usr/bin]
argv_count=7[/bin]
argv_count=8[/usr/bin/X11]
argv_count=9[/usr/X11R6/bin]
argv_count=10[/usr/games]
argv_count=11[/opt/bin]
argv_count=12[/usr/lib/mit/bin]
argv_count=13[/usr/lib/mit/sbin]
argv_count=14[/u01/mysql/share/bin]
  

But for bash  4.3 and 4.4(latest) the arguments passed to the first function 
call are not all divided at the IFS boundary.

In order to get all the arguments divided at the IFS boundary the arguments 
must first be concatenated into a variable and the variable used as the 
argument to the "doit" function. 

Here are the test Bash 4.3 results.

BASH_VERSION=4.3.42(1)-release
first_arg=1 
RESULT_A=`doit "$first_arg":2:3:4:$PATH\'
$RESULT_A=
doit argc=9
agv1=[1  2 3 4 /u01/mysql/5_7_19/usr/bin]
argv_count=2[/usr/local/bin]
argv_count=3[/usr/bin]
argv_count=4[/bin]
argv_count=5[/usr/bin/X11]
argv_count=6[/usr/X11R6/bin]
argv_count=7[/usr/games]
argv_count=8[/opt/bin]
argv_count=9[/u01/mysql/share/bin]

RESULT_B=`doit $all_args\'
$RESULT_B=
doit argc=13
agv1=[1 ]
argv_count=2[2]
argv_count=3[3]
argv_count=4[4]
argv_count=5[/u01/mysql/5_7_19/usr/bin]
argv_count=6[/usr/local/bin]
argv_count=7[/usr/bin]
argv_count=8[/bin]
argv_count=9[/usr/bin/X11]
argv_count=10[/usr/X11R6/bin]
argv_count=11[/usr/games]
argv_count=12[/opt/bin]
argv_count=13[/u01/mysql/share/bin]


Here is the test Bash 4.4(latest) results

BASH_VERSION=4.4.0(1)-release
first_arg=1 
RESULT_A=`doit "$first_arg":2:3:4:$PATH\'
$RESULT_A=
doit argc=10
agv1=[1  2 3 4 /usr/local/bin]
argv_count=2[/usr/bin]
argv_count=3[/bin]
argv_count=4[/usr/bin/X11]
argv_count=5[/usr/X11R6/bin]
argv_count=6[/usr/games]
argv_count=7[/opt/bin]
argv_count=8[/usr/lib/mit/bin]
argv_count=9[/usr/lib/mit/sbin]
argv_count=10[/u01/mysql/share/bin]

all_args="$first_arg:2:3:4:$PATH"
RESULT_B=`doit $all_args`
$RESULT_B=
doit argc=14
agv1=[1 ]
argv_count=2[2]
argv_count=3[3]
argv_count=4[4]
argv_count=5[/usr/local/bin]
argv_count=6[/usr/bin]
argv_count=7[/bin]
argv_count=8[/usr/bin/X11]
argv_count=9[/usr/X11R6/bin]
argv_count=10[/usr/games]
argv_count=11[/opt/bin]
argv_count=12[/usr/lib/mit/bin]
argv_count=13[/usr/lib/mit/sbin]
argv_count=14[/u01/mysql/share/bin]

As has been demonstrated the Bash 4.[3|4] test will only divide all the 
arguments at the IFS boundary when the function call is made with a single 
variable.


        

Glenn McCue

Shared Services Canada / Services partagés Canada 
8 Colonnade, Room 3020,
Ottawa, ON, K1A 0K9
Address Locator: 6901 

Phone:  (613) 220-5329
Email: glenn.mccue@canada.ca 
Fax: (613) 957-9942