Re: Segmentation fault when -x is added and variable contains nulls

[Greg Wooledge]

On Thu, Feb 06, 2014 at 11:51:25AM +0800, Dan Jacobson wrote:
> # su - nobody
> No directory, logging in with HOME=/
> $ cat /tmp/r
> LC_CTYPE=zh_TW.UTF-8 N=$(echo ??????|iconv -t big5 -f utf-8) sh -xc ': $N'
> $ sh /tmp/r
> /tmp/r: line 1:  4551 Segmentation fault      LC_CTYPE=zh_TW.UTF-8 N=$(echo 
> ??????|iconv -t big5 -f utf-8) sh -xc ': $N'
> 
> Something about that embedded null.
> bash, version 4.3.0(1)-rc1 (i486-pc-linux-gnu)

Just for the record, a variable can't contain a NUL byte.  I don't
know what your input string is (my system can't handle this locale; I
just see ?????? in my mail client), nor do I know what the output of
the iconv command is for that input.  Let's speculate that (as implied
by your Subject) there is a NUL byte somewhere in the middle of it.

First thing you should do is hunt down whoever invented a locale with
NUL bytes in characters, because that's just horrible.

Second thing, if the iconv command truly generates characters with a NUL
in them, then the variable N which is populated by reading this stream is
either going to be truncated at that point, or the NUL byte will simply
be skipped, depending on what your system's "sh" is.

In bash:

imadev:~$ N=$(printf '\x22\x23\x00\x24\x25')
imadev:~$ printf %s "$N" | od -t x1
0000000   22  23  24  25
0000004

If that NUL byte was required to form the correct character in your
locale, then the variable's actual contents (missing the NUL byte)
may be nonsensical.  That may be what's causing the seg fault.

If I were you, I'd check the output of that iconv command, hex dumping
it to get the exact sequence of bytes.  If one of them really is a NUL
byte then you can't store that sequence in a shell variable, or pass it
as an argument to any command.  You'll either have to store the data in
files, or switch to a language that can handle strings with embedded
NULs.