Re: Examples of concurrent coproc usage?

[Martin D Kealey]

On Wed, 17 Apr 2024, Chet Ramey wrote:
> On 4/16/24 2:46 AM, Carl Edquist wrote:
>
> > But the shell is pretty slow when you ask it to shovel data around like
> > this.  The 'read' builtin, for instance, cautiously does read(2) calls of a
> > single byte at a time.
>
> It has to do it that way to find the delimiter on a non-seekable file
> descriptor, since it has to leave everything it didn't consume available
> on stdin.

Has anyone tried asking any of the kernel teams (Linux, BSD, or other) to
add a new system call such as readln() or readd()?

I envisage this working like stty cooked mode works on a tty, except it
would also work on files, pipes, and sockets: you'd get back *at most* as
many bytes as you ask for, but you may get fewer if a delimiter is found.
The delimiter is consumed (and returned in the buffer), but everything
following a delimiter is left available for a subsequent read.

For a tty, code like

 tc_setattr(fd, &(struct termios){ .c_iflags = ICANON, .c_cc = { [EOL] = '\n' 
}});
 ssize_t n = read(fd, &buf, sizeof buf);

could become just

 ssize_t n = readd(fd, &buf, sizeof buf, "\n", 1, 0 /*flags*/);

or perhaps even

 ssize_t n = readdv(fd,
                    &buf, sizeof buf,
                    (struct iovec[]){{"\n", 1}}, 1,
                    0 /*flags*/);

I'm not sure whether multi-byte delimiters should be allowed, as it's
unclear what to do when you get an incomplete delimiter at the end of the
buffer, but an iovec interface would at least allow that as a future
possibility.

As Linux kernel developers have found, it's better to *always* include a
flags argument, even if you can't think of a use for it yet; but in this
case O_NONBLOCK and O_PEEK could immediately be useful.

-Martin

PS: I initially wondered about having

 ssize_t n = readvdv(fd,
                     (struct iovec[]){{&buf, sizeof buf}}, 1,
                     (struct iovec[]){{"\n", 1}}, 1);

but a vectored read isn't much of a saving unless the overall size is very
large, and enormous reads probably shouldn't use this facility since the
in-kernel byte scan would potentially block reads and writes by other
processes.

For the same reason I would allow the read size to be silently capped at a
value chosen by the kernel, probably a small multiple (1?) of the IO block
size or the pipe buffer size or the tty input buffer size.