Re: Why does "mapfile -d delim" (delim != '\n') use unbuffered read?

[Koichi Murase]

2021年5月8日(土) 3:30 Chet Ramey <chet.ramey@case.edu>:
> On 5/7/21 1:32 PM, Koichi Murase wrote:
> >> I don't think this will make much of a difference -- it never has,
> >
> > I suspect it's because the buffered read was only enabled when delim
> > == '\n' before and read(2) has read exactly until the delimiter in
> > most cases. Now we have changed that condition, so I guessed we want
> > to be more careful.
>
> I doubt it. The most common case is using delim == '\n' with a regular
> file, in which case the zread calls will buffer far beyond the end of
> the line. The zsyncfd fixed things up, but not until after the callback
> execution. Since I never received any bug reports about that behavior, I
> conclude that this change is not going to make much of a difference.

Ah, I see it was based on the fact that there have been no bug
reports. Thank you for the explanation. After some experiments, I
noticed that the typical use of `read' inside the callback doesn't
make difference because it also becomes the buffered mode and shares
the buffer with `mapfile'. However, something strange happens when
`read' is performed in another process or when some external program
reads the same fd.

For example, the following commands had resulted in an infinite loop
before the fix 59c575fd (commit "changes to mapfile, read builtins for
unbuffered reads"). Now it is fine:

$ printf "%s\n" {000001..00020} > tmp
$ callback() { x=$(read line;echo $line); }
$ mapfile -C callback -c 1 -t a < tmp

Other examples that make differences (unpredictable before 59c575fd,
but now predictable and well-defined):

$ callback() { read -d "" -n 7 line; echo "mapfile a[$1]=$2; callback $line"; }
$ callback() { echo "mapfile a[$1]=$2; callback $(bash -c 'read
line;echo $line')"; }
$ callback() { echo "mapfile a[$1]=$2; callback $(stdbuf -oL head -1)"; }

But I think these usages were not so common, so not many users would
have experienced the problem. To begin with, I guess there have been
not so many real use cases of the mapfile callback compared to that
without callback. Actually, I'm wondering in what situation we can
usefully specify the `-C callback' option because it seems like we can
anytime process blocks of elements after the entire array is loaded.
Maybe a replacement for `xargs' to limit the number of arguments
(which is more used for pipes than for regular files)?