Re: "here strings" and tmpfiles

[Daniel Kahn Gillmor]

On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote:
> On 4/9/19 2:56 AM, Jason A. Donenfeld wrote:
>> Since originally raising this issue with dkg (leading to this email
>> thread), I've only followed along from a bit of a distance. But it does
>> look like there's been some good progress: there's now a commit that
>> fills the pipe up to the OS's maximum pipe size, and then falls back to
>> the old (buggy, vulnerable, scary) behavior. 
>
> This is unnecessary hyperbole. The existing file-based mechanism works
> just fine. We're talking about what's essentially an optimization.

I think we all agree that avoiding the filesystem where possible is
likely to be an optimization and improvement (it means the heredoc will
work in some circumstances where it didn't work before, and it means not
having to juggle tmpfile names).

But as you said upthread, writing to disk is also a minor risk:

>>> It's a risk that most shells and shell users accept, and have for many
>>> years. That doesn't suggest there's no risk, but that it's minor.

I think Jason is conerned about the risk specifically, and maybe doesn't
care as much about the optimization.  I happen to care about both :)

In particular, i care about the heredoc/herestring persistence risk
because i know for a fact that some people don't understand that
heredocs and herestrings aren't likely to be ephemeral, and have been
using them with the intent of ephemerality.  I've personally harbored
that misunderstanding in the past, and i'm pretty sure that there are
other people using bash who are even less sophisticated about what's
going on under the hood than i am.  I'd love to have bash protect those
users from themselves at some point in the future automatically :)

>>   - The security of this language construct is now OS and runtime-
>>     configuration dependent. That means it's not that reliable, and so
>>     we're basically back at advising square one: "don't use herestrings".
>
> This doesn't make any sense.

If we look at the problem from the perspective of the risk of
herestring/heredoc content leaking to non-ephemeral storage, then
i think Jason's perspective makes sense.  he's asking "how can we advise
users of bash about the ephemerality of herestrings/heredocs?"  And if
the answer is "it depends…" then the safe+simple guidance is "you MUST
NOT assume that herestrings/heredocs are ephemeral".

>> A real solution for this issue involves getting rid of the temporary file
>> all together. Since we're talking about a bash string, it's already in
>> memory. Why not just fork() if the write() will block? A simple way would be
>> to always fork(). A fancy way would be to set NONBLOCK mode, see if it
>> returns EAGAIN, and only fork() if the write would block. Either way seem
>> basically fine, with the critical part being that the temporary file is
>> totally gone from the equation.
>
> If someone would like to take the code in the devel branch and add this, I
> would certainly look at it.

Either of these implementations would be great, though i note that how
bash handles the extra fork (whether the parent or the child does the
pipe writing) might complicate the use of the wait builtin (or any
wait(-1) syscall) in the spawned process (e.g. similar to this bug
report: https://bugs.debian.org/920455)

Anyway, if we had either fix in place, then the safe+simple guidance
about ephemerality would be "as long as you're using bash version >=
$SOME_VERSION, you can count on heredocs/herestrings being ephemeral",
which would be super nice.

Thanks for all the work on bash!

      --dkg

signature.asc
Description: PGP signature