Re: Examples of concurrent coproc usage?

[Carl Edquist]

On Mon, 15 Apr 2024, Zachary Santer wrote:

> As much as you can just printf the same thing once for each fd, that 
> doesn't work super well for binary data.

Well, you _can_ shovel binary data too: (*)

        while IFS= read -rd '' X; do printf '%s\0' "$X"; done

and use that pattern to make a shell-only version of tee(1) (and I suppose 
paste(1)).  Binary data doesn't work if you're reading newline-terminated 
records, because you cannot store the NUL character in a shell variable. 
But you can delimit your records on NULs, and use printf to reproduce 
them.

But the shell is pretty slow when you ask it to shovel data around like 
this.  The 'read' builtin, for instance, cautiously does read(2) calls of 
a single byte at a time.  And printf will write(2) each null-terminated 
recored unbuffered.  Meanwhile cat(1) and tee(1) (which read and write 
larger blocks) are much faster.

But as I said before, the main work of the shell is making it easy to set 
up pipelines for other (very fast) programs to pass their data around.


[(*) It's slow compared to cat(1) or cp(1), but it does work.  I had do 
resort to this once in a botched glibc system upgrade that had wrecked my 
libc symlinks.  A broken libc symlink meant none of the external commands 
worked at all, so I had to fix everything with only the builtin commands 
of my existing live root shell.  I was able to copy all the (binary) .so 
files to another directory (/dev/shm) using the above read/printf loop, 
with the names fixed, then export LD_LIBRARY_PATH to point to the new 
copies.  That then made it possible to run external commands again (ln(1), 
in particular) to actually fix the symlinks.]


> I've thought about splitting and recombining pipelines like this, but 
> I've never had a reason to.

"If you build, it they will come."  ;)


Carl