Re: "here strings" and tmpfiles

[Chet Ramey]

On 4/10/19 11:59 AM, Daniel Kahn Gillmor wrote:
> On Wed 2019-04-10 09:07:27 -0400, Chet Ramey wrote:

> I think we all agree that avoiding the filesystem where possible is
> likely to be an optimization and improvement (it means the heredoc will
> work in some circumstances where it didn't work before, and it means not
> having to juggle tmpfile names).
> 
> But as you said upthread, writing to disk is also a minor risk:
> 
>>>> It's a risk that most shells and shell users accept, and have for many
>>>> years. That doesn't suggest there's no risk, but that it's minor.

OK, so take a risk management approach. That risk is so minor that it has
existed -- and been accepted -- for as long as here documents have existed.
That risk I mentioned is not just security-related, though I imagine that's
where Jason's perspective is concentrated, but concerns resource shortages
(full disks) and availability (no writable file systems) as well. There is
such a thing as negligible risk, or risk that's easy to mitigate.

> 
> I think Jason is conerned about the risk specifically, and maybe doesn't
> care as much about the optimization.  I happen to care about both :)

Is it just that people have not realized all along that most shells,
certainly all historical shells, that implement here documents use temp
files to do it? It's really only the ash-based shells (not an insignificant
portion of the shells in use, for sure) that use pipes exclusively.

> 
> In particular, i care about the heredoc/herestring persistence risk
> because i know for a fact that some people don't understand that
> heredocs and herestrings aren't likely to be ephemeral, and have been
> using them with the intent of ephemerality.  I've personally harbored
> that misunderstanding in the past, and i'm pretty sure that there are
> other people using bash who are even less sophisticated about what's
> going on under the hood than i am.  I'd love to have bash protect those
> users from themselves at some point in the future automatically :)

I'd love to see these concerns articulated concretely, so we can analyze
the risk in terms of the temp-file-on-disk implementation.

>>>   - The security of this language construct is now OS and runtime-
>>>     configuration dependent. That means it's not that reliable, and so
>>>     we're basically back at advising square one: "don't use herestrings".
>>
>> This doesn't make any sense.
> 
> If we look at the problem from the perspective of the risk of
> herestring/heredoc content leaking to non-ephemeral storage, then
> i think Jason's perspective makes sense.  he's asking "how can we advise
> users of bash about the ephemerality of herestrings/heredocs?"  And if
> the answer is "it depends…" then the safe+simple guidance is "you MUST
> NOT assume that herestrings/heredocs are ephemeral".

I'm sure Jason has been advocating that nobody use here documents since the
Bourne shell introduced them, since that's the position he advocates above.
Unnecessary hyperbole diminshes the force of your argument.

The real question is why the ephemerality of heredocs and herestrings is
important -- what is the use case that requires it, as opposed to it simply
being an implementation detail?


>> If someone would like to take the code in the devel branch and add this, I
>> would certainly look at it.
> 
> Either of these implementations would be great, though i note that how
> bash handles the extra fork (whether the parent or the child does the
> pipe writing) might complicate the use of the wait builtin (or any
> wait(-1) syscall) in the spawned process (e.g. similar to this bug
> report: https://bugs.debian.org/920455)

It wouldn't really affect that. The reason `wait' waits for process
substitution processes is that they set $!, making them "known to the
shell" and subject to wait without arguments.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/wait.html#tag_20_153

> 
> Anyway, if we had either fix in place, then the safe+simple guidance
> about ephemerality would be "as long as you're using bash version >=
> $SOME_VERSION, you can count on heredocs/herestrings being ephemeral",
> which would be super nice.

What's gained by making that guarantee that was keeping people from using
here documents before? I'm genuinely curious about use cases.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

signature.asc
Description: OpenPGP digital signature