Re: Examples of concurrent coproc usage?

[Carl Edquist]

On Mon, 22 Apr 2024, Martin D Kealey wrote:

> On Sun, 21 Apr 2024, 10:13 Carl Edquist, <edquist@cs.wisc.edu> wrote:
>
> > You mean, specifically in order to implement a slightly-more-efficient 
> > 'read' builtin in the shell?
>
> The read built-in in the shell is only one case that would benefit from 
> such a syscall.
>
> The purpose would be to allow multiple processes to read in turn from a 
> consumable (or otherwise non seekable) input stream. In this context 
> doing a large block read() is exactly what we DON'T want to do, so we 
> also can't use a library function such as getline() that is built on top 
> of such a read().
>
> By way of example, another use would be the "head" utility, which by 
> using such a syscall could consume only the bytes it outputs, leaving 
> all other bytes still in the input stream. This would be an improvement 
> over the current situation.
>
> Basically any time you have cooperating processes reading delimited 
> input, this would be an improvement.

Makes sense!


> > I envisage this working like stty cooked mode works on a tty,
>
> …
>
> > One downside is you'd end up with a system call for each token
>
>
> That's not how stty cooked mode normally works.
>
> The typical use case is line-at-a-time, so this would reduce the number 
> of system calls by about 90% on a typical text input stream, more if 
> there are few or no blank lines.
>
> However I would not hard code "newline" into the kernel, but rather 
> allow the user code to nominate a list of delimiters.

When I say "token" I just mean a record with whatever delimiter you're 
using.  Assuming the reading stops after consuming the first delimiter 
(which is necessary for the 'read' builtin), then you end up with one 
system call per line or record or token or whatever you want to call it.


A line at a time is an improvement over a byte at a time, but if/when you 
can find a way to do what you want with the default block buffering & 
userspace tokenizing, you'll reduce the number of system calls by 99.99%. 
(Which is one reason I was saying the shell is crippled when limited to 
builtins; eg, a read/printf loop compared to simply running cat.)

...

But yeah currently a pipe with a series of records and multiple 
cooperating/competing readers perhaps only works if the records have a 
fixed size.  A new readd[elim] system call like you're talking about would 
allow safely reading a single variable-length record at a time.

So by all means, feel free to take the idea to your friendly neighborhood 
kernel community, if you'd like to pursue it  ;)


Carl