Re: bash uses tmp files for inter-process communication instead of pipes?

[Linda Walsh]

Greg Wooledge wrote:
> On Mon, Oct 06, 2014 at 12:38:21PM -0700, Linda Walsh wrote:
> > According to Chet , only way to do a multi-var assignment in bash is
> >
> > read a b c d  <<<$(echo ${array[@]})
>
> The redundant $(echo...) there is pretty bizarre.  Then again, that
> whole command is strange.  You have a nice friendly array and you are
> assigning the first 3 elements to 3 different scalar variables.  Why?
> (The fourth scalar is picking up the remainder, so I assume it's the
> rubbish bin.)
>
> Why not simply use the array elements?
>
> > Forcing a simple assignment into using a tmp file seems Machiavellian --
> > as it does exactly the thing the user is trying to avoid through
> > unexpected means.
>
> You are working in a severely constrained environment. 
That isn't the problem:  the assignment using a tmp file is:
> strace -p 48785 -ff
Process 48785 attached
read(0, "\r", 1)                        = 1
write(2, "\n", 1)                       = 1
socket(PF_NETLINK, SOCK_RAW, 9)         = 3
sendmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, 
msg_iov(2)=[{"*\0\0\0d\4\1\0\0\0\0\0\0\0\0\0", 16}, {"read a b c d 
<<<${arr[@]}\0", 26}], msg_controllen=0, msg_flags=0}, 0) = 42
close(3)                                = 0
-----
Um... it used a socket.. to transfer it, then it uses a tmp file on top
of that?!  :

rt_sigaction(SIGINT, {0x4320b1, [], SA_RESTORER|SA_RESTART, 0x30020358d0}, 
{0x4320b1, [], SA_RESTORER|SA_RESTART, 0x30020358d0}, 8) = 0
open("/tmp/sh-thd-110678907923", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
write(3, "one two three four", 18)      = 18
write(3, "\n", 1)                       = 1
open("/tmp/sh-thd-110678907923", O_RDONLY) = 4
close(3)                                = 0
unlink("/tmp/sh-thd-110678907923")      = 0
fcntl(0, F_GETFD)                       = 0
fcntl(0, F_DUPFD, 10)                   = 10
fcntl(0, F_GETFD)                       = 0
fcntl(10, F_SETFD, FD_CLOEXEC)          = 0
dup2(4, 0)                              = 0
close(4)                                = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 
0x7fff85627820) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(0, 0, SEEK_CUR)                   = 0
read(0, "one two three four\n", 128)    = 19
dup2(10, 0)                             = 0
fcntl(10, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
close(10)                               = 0
-----

Why in gods name would it use a socket (still of arguable overhead, when
it could be done in a local lib), but THEN it duplicates the i/o in a file?

> Thus, you need
> to adapt your code to that environment.  This may (often will) mean
> you must forsake your usual practices, and fall back to simpler
> techniques.
----
        The above is under a normal environment.  It's still broken.




> Maybe the best solution here is to move your script to a different part
> of the boot sequence.  If you run it after all the local file systems
> have been mounted, then you should be able to create temporary files,
> which in turns means << and <<< become available, should you need them.
----
        Theoretically, they ARE mounted.... What I think may be happening
is that $TMP is not set so it is trying to open the tmp dir in:

"//sh-thd-183928392381" -- a network address.


> Elegance must be the first sacrifice upon the altar, here.
---
        Correctness before elegance.  1), use memory before OS services.
2) use in-memory services before file services, 3) Don't use uninitialized
variables (TMP) -- verify that they are sane values before usage.
4) don't use network for tmp when /tmp or /var/tmp would have worked just
fine.



> > So why would someone use a tmp file to do an assignment.
>
> It has nothing to do with assignments.  Temp files are how here documents
> (and their cousin here strings) are implemented.  A here document is a
> redirection of standard input.  A redirection *from a file*
----
        Nope:
 This type of redirection instructs the shell to  read  input  from  the
       current source until a line containing only delimiter (with no trailing
       blanks) is seen.  All of the lines read up to that point are then  used
       as the standard input for a command.

"The current source" -- can be anything that input can be
redirected from -- including memory.


> in fact,
> albeit a file that is created on the fly for you by the shell.
----
Gratuitous expectation of slow resources...  Non-conservation
of resources, not for the user, but for itself.

Note:
> a=$(<foo)
> echo "$a"
one
two
three
---
no tmp files used, but it does a file read on foo.

b=$a
-- the above uses no tmp files.

b=$(echo "$a")
---
THAT uses no tmp files.

But
b=<<<$a

uses a tmp file.

That's ridiculously lame.





> > So why would a temp file be used?
>
> Historical reasons, and because many processes will expect standard input
> to have random access capabilities (at least the ability to rewind via
> lseek).  Since bash has no idea what program is going to be reading the
> here document, why take chances?
----
        How could you use a seek in the middle of a HERE doc
read?

Not only that, it's just wrong.    /dev/stdin most often comes from
a tty which is not random access.


> > Creating a tmp file to do an assignment, I assert is a bug.
>
> You are not *just* doing an assignment.  You are doing a redirection.
> Furthermore, you have not even demosntrated the actual reason for the
> here document yet.
----
        I
> > cmd1 | cmd2 -- that hasn't used tmp files on modern *nix systems for
> > probably 20 years or more (I think DOS was the last shell I knew that used
> > tmp files...)
>
> Correct.  Pipelines do not use temp files.
>
> > so why would "cmd2 < <(cmd1 [|])" not use the same paradigm -- worse, is
>
> Process substitution uses either a named pipe, or a /dev/fd/* file system
> entry, or a temp file, depending on the platform.
---
        I didn't ask *what*  -- I asked *why*.
Why is  different mechanism used for the same function.

you are taking output of 1 process and redirecting it's output into the input
of another... They are the same.


> I don't know which one your *real* script is using, so it's hard to
> advise you.
----
How much of the original do you want?... wait....... um...
But the point it it should already work... I think it is trying to read
from the network.

shopt expand_aliases
alias dcl=declare         sub=function
alias int=dcl\ -i         map=dcl\ -A       hash=dcl\ -A    array=dcl\ -a
alias lower=dcl\ -l       upper=dcl\ -u     string=dcl      my=dcl
sub get_net_IFnames_hwaddrs () {        # get names + addrs from /sys
  vrfy_drivers
  array pseudo_devs=(br bond ifb team)
  string pseudo_RE="^+(${pseudo_devs[@]})$"
  pseudo_RE=${pseudo_RE// /|}
  string netdev_pat="+([_0-9a-z])+([0-9])"
  sysnet=/sys/class/net
  ( cd "$sysnet" &&
    for nm in $( printf "%s\n" $netdev_pat | grep -Pv "$pseudo_RE"); do
      echo "$nm" "$(<$nm/address)"
    done )
}

map act_hw2if                           #actual values (to be read in)
map act_if2hw
map XIF                                 #tmp array to hold exchanged IF's

sub read_actuals () {                   # parse output stream from above
  my ifname hwaddr
  while read ifname hwaddr; do
    act_hw2if[$hwaddr]="$ifname"
    act_if2hw[$ifname]="$hwaddr"
  done <<<"$(get_net_IFnames_hwaddrs)"
}