[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Proposed new feature for bash: unbuffered pipes

From: Daniel Colascione
Subject: Re: Proposed new feature for bash: unbuffered pipes
Date: Thu, 23 Apr 2020 17:26:56 -0700
User-agent: AquaMail/1.24.0-1572 (build: 102400003)

On April 22, 2020 6:32:07 PM address@hidden (Dale R. Worley) wrote:

The crux of the problem, IMHO, is to look at it from the right angle:

Occasionally, the user desires that I/O through certain pipes should be
unbuffered, that is, the stdio stream(s) that write into the pipe should
be unbuffered, rather than the default block-buffered.  These are
situations where the user can tell that it is desirable to sacrifice
efficiency to gain low latency, whereas the writer of the program that
the user is running could not predict the needs of this particular
invocation.  (This is where Ulrich Drepper was incorrect; the writer of
the program doesn't know when this is desirable.)

This facility is best activated by a feature in bash, as the need arises
from the relationship between two processes.

The ideal way to implement this would be for the kernel to provide two
flavors of pipe, one designated buffered and one designated unbuffered,
and according to the user's instructions, bash would allocate an
unbuffered pipe when it was needed.  The kernel itself would treat the
two types of pipe in the same way, but stdio would create buffered
streams for fd's that were opened on buffered pipes and it would create
unbuffered streams for fd's that were opened on unbuffered pipes.

However, getting a coordinated change into the kernel seems like it
would be quite difficult.

So an alternative implementation is to have a way for bash to notify
processes it creates that certain pipes (identified by their dev/ino
numbers) should have streams that are unbuffered.  Pretty much the only
way to do this is to have a conventional environment variable for
communicating this information.

Of course, either of these solutions requires changes to glibc, but
then, if you're going to modify buffering behavior of streams, you are
going to have to modify glibc.

Given the current political realities in the relevant FOSS ecosystem, it might be easier to add a new pipe flavor or attribute to the kernel and then get libc implementations to support it as a fait accompli than to do something purely in libc. Also, an environment variable is semantically wrong: its scope is too broad, well beyond the specific pipe you care about. Imagine the variable leaking into a daemon that lives forever.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]