[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: redirecting a file descriptor to an array variable? Possible? How? R

From: Linda Walsh
Subject: Re: redirecting a file descriptor to an array variable? Possible? How? RFE?
Date: Wed, 18 Nov 2015 10:16:46 -0800
User-agent: Thunderbird

Greg Wooledge wrote:
On Sun, Nov 15, 2015 at 05:26:26AM -0800, Linda Walsh wrote:
This can be used to serialize an array into a file
and to read it back:

printf '%s\0' "address@hidden" > file
        So just like you can have [<>|]&[-0-9][0-0] as redirection ops that work
with files or FD's, why not have something like:

printf '%s\0' "address@hidden" >&address@hidden (**)

**: or ">@out" or ">&@out"

using 4.3,   But how is this:

  > mapfile -td '' array < file

That's fine for reading files, but I'm wanting this to be doable
in the absence of a file system, so again, instead of '>' used in printf,
above, use '<' -- but have the object it reads from represent
an array -- so you could have your file in an 'array' with the
nul's trimmed off (or nl's).

As for not wanting to use files... they are an unnecessary
dependency in most cases and file opens are expensive on some
OS's -- more so than linux (ex. cygwin).

If I have GigaBytes of memory, why should I have to use a file system to store transient, tmp information (unless it exceed all of memory), BUT -- that has often happened when "/tmp" hasn't been
large enough -- with cheaper memory, note the situation on my system
as it's been for a couple of years now...

df /tmp  /dev/mem
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc2       7.8G  3.3G  4.6G  42% /tmp
devtmpfs         48G   44K   48G   1% /dev

So if you are writing to a tmp file, which will die of
ENOMEM first?

If I thought adding 4.6G to '48G' (statistically), enough
of the the time, I'd certainly arrange for a spill.

But at run time, wouldn't it be smart to check resources --
like disk space in /tmp or free memory?  If free memory is
10x the disk space, maybe using the memory should be given
priority in automatic mode, thought for corner cases, 'tmp' should allow "auto_largest", "mem", "/tmp" or "~/tmp", or auto_besteffort (do what one can -- but intially, start w/auto_largest,
then add a spell-to-disk "/tmp", if there is <90% space on it...

        But contrary to the manpage under printf:
" The -v option causes the  output  to  be
 assigned  to  the  variable var rather than being printed to the
 standard output. "

printf does not print the output of the above into a var instead
of stdout.
Seems like that would be a bug?

The command you used above does not have '-v var' in it.
1) printf '1\000\n'  | hexdump -C
00000000  31 00 0a                                          |1..|

2) printf '1%c\n' '\x00' | hexdump -C 00000000 31 5c 0a |1\.|

1) printf -v v '1\000\n'
declare -p v
declare -- v="1"      <<-- note that $v does not have '1\000\n' in it.


1) printf '"%c"\n' '' | hexdump -C 00000000 22 00 22 0a 1b) printf '"%c"\n' $'\x00' | hexdump -C 00000000 22 00 22 0a # Showing, indeed, '' == $'\x00', in this case, however, if we
try printing the result using "-v v" and look at it:

1st, assignment:
unset v; printf -v v '"%c"\n'  $'\x00' ;declare -p v
declare -- v="\""        <<-- again note that "$v" doesn't contain $'"\x00"'.

If I am in a locale using UTF-16, there will be lots of 'NUL'
chars if I view that output as binary -- so there has to be one
or more ways to encode NUL bytes in a var such that they don't
terminate the string.

I do not have access to a UTF-16 environment.  Does bash even WORK
in such an environment?  At all?
        Actually, on this one, I'll have to find a set of 'locale'
files to describe the charset.  Several files that work with the
"LC_" files end up working due to the way they are specified ..
for example, the 1st go-around for the Chinese to encode their stuff
and everything else ended up considerably larger than UTF-8:

/usr/share/i18n/charmaps> ll --sort size -r | tail -2
-rw-rw-r-- 1 1951778 Aug 11 11:55 UTF-8
-rw-rw-r-- 1 4181789 Nov 17 15:12 GB18030

GB18030 _does_ provide a 1:1 mapping of Unicode codepoints there and
back again (same as UTF-8 and UTF-16). But since UTF-16 & UTF-8 have the same # of codepoints, they'd
likely be similar in size.

I don't see the relation between these two issues.  "Being able to
store NUL bytes in a string" has nothing to do with "being able to
capture stdout and stderr separately in shell variables" as far as
I can see.
        As I pointed out above, you cannot safely create, save or restore
output from printf that will be redirected into a variable, which
contradicts the manpage specification saying:

    The -v option causes the  output  to  be
    assigned  to  the  variable var rather than being printed to the
    standard output.

Chet would have to introduce a new feature for the latter to be possible.
I'm sure it's not trivial to handle all the different surprise cases.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]