Re: "here strings" and tmpfiles

[Daniel Kahn Gillmor]

On Wed 2019-04-10 16:16:44 -0400, Chet Ramey wrote:
> Is it just that people have not realized all along that most shells,
> certainly all historical shells, that implement here documents use temp
> files to do it? It's really only the ash-based shells (not an insignificant
> portion of the shells in use, for sure) that use pipes exclusively.

Yes, i think most people have not realized that, all along.  I certainly
didn't, and i've been using bash for decades now.

> I'd love to see these concerns articulated concretely, so we can analyze
> the risk in terms of the temp-file-on-disk implementation.

I tried to do that at the start of this thread, but i'm happy to try
gain.

data written to the local filesystem can be discovered by someone
analyzing the disk controller data path, or by someone with access to
the underlying storage medium.

Even with encrypted block devices (e.g., dmcrypt), it's possible that
the key to the block device becomes known to the attacker, which allows
them to read the contents of blocks on the free list.

So anyone who puts anything sensitive into a here document or herestring
using bash is triggering a potential leak of that sensitive data.  While
the contents of many here documents are clearly not sensitive, or are
destined for the filesystem anyway, *some* data placed in heredocs *are*
sensitive, and are intended to be ephemeral.  Obvious examples include
cryptographic keying material or passwords known to the shell but not
currently on disk anywhere already.  But i'm sure there are many
non-obvious cases as well.  Part of the problem with evaluating this is
that there are many uses of bash that none of us on this list will ever
see.

> I'm sure Jason has been advocating that nobody use here documents since the
> Bourne shell introduced them, since that's the position he advocates above.

AFAICT, Jason only recently became aware of this, so he certainly hasn't
been advocating no one use here docuemnts  the Bourne
shell introduced them:

  
https://git.zx2c4.com/password-store/commit/?id=367efa5846492e1b0898aad8a2c26ce94163ba24

He's not the only person who has used them in an attempt at simplicity
or cleanliness only to find that this is a risk if sensitive data is
included in them.

> Unnecessary hyperbole diminshes the force of your argument.

i wasn't intending to use unnecessary hyperbole -- if you want to give
guidance that most people can follow, it's necessary for the guidance to
be clear and simple.  Complicated guidance gets ignored, or people get
confused and can't follow it closely.

fwiw, i'm not here to have an argument with you or to "win" at anything
-- i'm trying to help improve bash and the ecosystem around it, the same
as you are.  I certainly have no interest in having this be a contest.
If you see this discussion as an argument, please let me know and i'll
stop following up on this thread and adjust my expectations accordingly.

> The real question is why the ephemerality of heredocs and herestrings is
> important -- what is the use case that requires it, as opposed to it simply
> being an implementation detail?

This is an implementation detail that isn't well-documented.  in the
initial report, part (d) suggests that if this is an intended
implementation detail, it ought to be called out explicitly in the
documentation, so that users at least have a chance of avoiding this
kind of leakage.  (at least, those users who read the documentation, anyway)

>> Either of these implementations would be great, though i note that how
>> bash handles the extra fork (whether the parent or the child does the
>> pipe writing) might complicate the use of the wait builtin (or any
>> wait(-1) syscall) in the spawned process (e.g. similar to this bug
>> report: https://bugs.debian.org/920455)
>
> It wouldn't really affect that. The reason `wait' waits for process
> substitution processes is that they set $!, making them "known to the
> shell" and subject to wait without arguments.
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/wait.html#tag_20_153

This behavior is actually different between bash 4.4.18 and 5.0, but i
think this is a separate discussion, so i'll defer it to a different
thread to avoid confusion here :) 

> What's gained by making that guarantee that was keeping people from using
> here documents before? I'm genuinely curious about use cases.

The security use case is for handling data that may be sensitive and
ephemeral.  Not every program using bash even knows whether the data it
has access to is intended to be ephemeral (and bash certainly doesn't
know), so unless the operator is explicitly aiming to write data to the
disk, it would be great to avoid touching non-volatile storage.

I hope this helps in understanding this different perspective somewhat.

all the best,

    --dkg

signature.asc
Description: PGP signature