Re: bash 5.1 heredoc pipes problematic, shopt needed

[Chet Ramey]

On 4/28/22 10:39 PM, Ángel wrote:
> On 2022-04-25 at 10:40 -0400, Chet Ramey wrote:
> > > > Perhaps BASH_COMPAT=5.0 could be extended to handle this.
> > >
> > > I think a shopt makes more sense. Forcing heredocs to be files
> > > although something legit to request, is more a caller workaround to
> > > bugs in called programs.
> >
> > Shell options aren't intended to cater to broken programs; that's
> > what the compatibility mode is for.
> >
> > It's not `legit to request'; the only thing guaranteed is that the
> > user will get a file descriptor they can read from.
>
> This response originally surprised me, but you have a point. The
> interface is just a fd, and usage of a file an implementation detail.
> Still, based on what surfaced on this thread, one might argue the need
> for a *seekable* fd, in which case we end up in a pretty similar
> situation.

https://lists.gnu.org/archive/html/bug-bash/2019-03/msg00082.html

> A seekable fd could be made with Linux-specific memfd_create(), maybe
> worth considering.

You can't make that file descriptor read-only.

https://lists.gnu.org/archive/html/bug-bash/2019-03/msg00083.html


> I see a pattern there where a heredoc is converted to a fd (pipe/file),
> then the fd is (often) made into a bash buffered input. I think that
> going directly from heredoc to the buffered input would allow avoiding
> the temporary file/pipe altogether, and the penalty associated with 1-
> byte reads in the case where it is not going to a separate program.
> That probably requires some refactoring to work out though.

They're two separate operations, and have to be kept separate semantically
to be compatible with historical usage. It boils down to the read builtin
recognizing when the file descriptor it's using for input is a here-
document and using a separate internal buffer, right? Is that much of a
special case worth the code it would take to implement?

> PS: Looking at the use_tempfile branch (actually the part fulfilled by
> sh_mktmpfd), that could benefit from O_TMPFILE | O_EXCL on modern Linux
> kernels.

https://lists.gnu.org/archive/html/bug-bash/2019-03/msg00091.html

You can't make that file descriptor read-only either (nor is it portable).

My preference is a portable memfd_create(); I think I could get around
its limitations.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/