Re: "here strings" and tmpfiles

[Chet Ramey]

On 3/18/19 5:18 PM, Daniel Kahn Gillmor wrote:
> hi bash developers--
> 
> I ran the following command to get a sense of how bash deals with here
> strings under the hood:
> 
>     strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"'
> 
> (i'm testing with bash 5.0-2 on debian testing/unstable).
> 
> It turns out that this creates a temporary file, actually touching the
> underlying filesystem:

Yes, that's how bash chooses to implement it. There are a few portable ways
to turn a string into a file descriptor, and a temp file is one of them (a
child process using a pipe is another, but pipes have other issues).


> I could find no mention in the bash(1) manpage of any risk of either
> here documents or here strings touching the underlying filesystem.

Why would the man page mention this?

> 
> I know that some systems use heredocs or herestrings explicitly to avoid
> things like:
> 
>  * writing to the filesystem,
>  * invoking extra processes, or
>  * making sensitive data avaialble to the process table.

These are making assumptions about the underlying implementation that are
not guaranteed.

> So writing this stuff to the filesystem, where it is likely to touch the
> underlying disk, seems particularly problematic. And of course there is
> the potential for weird race conditions around filename selection common
> to all tmpfile-style shenanigans.
> 
> A few possible options for trying to improve the situation:
> 
>  a) use socketpair(2) or pipe(2) instead of making a tmpfile.  this has
>     the potential downside that the semantics of access to the remaining
>     file descriptor would be subtly different from "regular file"
>     semantics.

Correct, plus a general implementation would require a child process to
send the data through the pipe.

> 
>  b) On systems that support O_TMPFILE, try something like
>     open("/dev/shm", O_RDWR|O_CREAT|O_EXCL|O_TMPFILE).  /dev/shm tends
>     to be a globally-writable tmpfs, so that avoids touching any disk,
>     and O_TMPFILE avoids tmpfile-style race conditions.  This might need
>     a fallback (to the current tmpdir selection mechanics?) in case
>     /dev/shm isn't available.
> 
>  c) Just use O_TMPFILE with the current tmpdir selection mechanics, if
>     it's supported.  This isn't quite as clever as trying to use
>     /dev/shm first, and it won't fix the herestrings hitting the disk,
>     but it at least avoids tmpfile races.

I prefer to support more portable alternatives. If someone wants to take a
run at an implementation that uses a pipe, I'd be happy to take a look at it.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

signature.asc
Description: OpenPGP digital signature