Re: Do `>& -` and `<& -` also work?

[Peng Yu]

On Wed, May 12, 2021 at 1:05 AM Koichi Murase <myoga.murase@gmail.com> wrote:
>
> 2021年5月11日(火) 13:06 Peng Yu <pengyu.ut@gmail.com>:
> > I totally understand what you are saying. But there are
> > POSIX-noncompliant shells that are also very popular. See the
> > following shell. It says "fish is ranked 2nd while bash is ranked
> > 3rd." I don't know how trustworthy that fish is more popular than
> > bash. At least, it should be very popular. So being POSIX-compliant or
> > not is not something dictate whether a shell is popular or not. In
> > this sense, completely POSIX-compliance is not that important.
>
> You have told me in another thread that you are not discussing the
> change to the existing code of Bash, but rather talking about
> hypothetical design when one would have implemented a new one from
> scratch based on the experience on Bash. If that is also the same for
> this discussion,

Yes.

> I think maybe you could have first clarified it

OK. I don't think I explicitly said I want to request changing the
current source code of bash. Consider whether some part of bash
(either the abstract syntax or the implementation of it) can be
improved is different from actually requesting changing it. People who
are confused about the two may be more careful at reading my text. If
there were indeed cases I explicitly requesting changing the bash
source code, please point it to me, I will be more careful about my
wording.

> before starting the discussion in order to avoid unnecessary conflicts
> something like "that's not the direction of Bash, etc."
>
> > I checked the syntax of fish, which seems to be too far away from the
> > bash syntax. I'd rather have something similar to bash but fixing the
> > bad designs over the last 40 years. For this reason, it would
> > beneficial to evaluate what syntax designs are good and what designs
> > are bad in bash (which can come from POSIX). If there were somebody
> > who starts with a better shell similar to bash, that could serve as a
> > reasonable good starting point.
>
> There are already many attempts for "better syntax" in shells. You can
> find a list of such shells (that are free from the constraints of the
> POSIX) at:
>
> https://github.com/oilshell/oil/wiki/Alternative-Shells

This list is long, but most of them are not relevant to me. The shell
that matters to me must be compatible to bash (at least for the
features that I commonly use as I don't think I know all the features
of bash, for example, I've never used BASH_XTRACEFD, let alone other
features that still don't know today). But that alternative just not
exist.

The statement "POSIX-ish shells like ... zsh" is wrong or not precise,
as zsh is not POSIX compatible by default as mentioned below.

https://wiki.archlinux.org/title/zsh

"Zsh ... While being compatible with the POSIX sh (not by default,
only if issuing emulate sh)"

> Some of them try to improve the Bash syntax,

To me, improve the syntax is important but secondary, I think that it
is to separate its parser with the backend so that optimization can be
done at the backend.

At this moment, it seems that in the bash code the front-end parsing
and back-end evaluation are tied together, making optimization not
easy to do. Maybe if an AST can be parsed, then a lot of new things
can be done with the bash language? For example, I am thinking whether
built-in program can be written in bash language instead of in C as
they are current written.

The analogy is like cython vs python. For the cython tutorial, it
looks like cython is very similar to python. Having this similarity
should be a boost to productively as people can write python code and
then make minor changes to get cython performance. Doing requires some
extension in cython. I think there are certain extensions for the bash
language to do something similar.

https://cython.readthedocs.io/en/latest/src/tutorial/cython_tutorial.html

> and some try to invent a
> completely different syntax.
>
> Also, the reason that fish became popular isn't its syntax. Fish is
> just popular for an interactive shell with eye candies like syntax
> highlighting and autosuggestions, and almost no one writes scripts in
> Fish (except for its configuration for interactive uses).

The syntax could be just its drawbacks. For example to write math, one
has to write something. I won't write any math in a script like this.
It is just too verbose.

fish$ set x 4
fish$ set y 7
fish$ set z (math $x + $y)

In bash, the code is the following, it is much more cleaner and intuitive.

((x=4,y=7,z=x+y))

Taking about math, one thing that could be done in bash is to have a
math mode like this, so that it allows many math operations. I think
it is not even impossible to implement a MATLAB syntax in this mode to
allow full-fledged math operations (of course matrix must be supported
as a datatype).

((
 x = 4
 y = 7
 z = x + y
))

> Actually,
> Fish has many serious defects in its syntax design which cannot be
> fixed for backward compatibility. Bash is far better than Fish as a
> scripting language. I guess it is very interesting for you to read
> GitHub issues of the Fish shell listed at
>
> https://github.com/pirate/fish-utils/blob/master/README.md

Got it.

> I think every language has such an aspect in syntax; there is always a
> way of writing that most people don't use but syntactically correct.
> For example, in C language, most tutorials and texts put spaces around
> operators like « a = b » but we can also write « a=b ».

This is not a comparible example to the case of >&-. Both "a = b" and
"a=b" are commonly seen in C. But I just don't remember I've seen ">&
-" in any tutorials when I first learn bash.

This is almost a philosophical question: Whether the evolution of
anything should be compatible to all past features, or just be
compatible with the majority of them. I know Windows takes the former,
which makes their API full of functions with number suffixes. Is it
good or bad? I think that it depends. For people who needs to learn
such an API, it is definitely bad, but for people who just run the
program, it is definitely good. So there is a subtle balance between
the two.

I do think there is no need to be 100% backward compatible. But what
not to be backward compatible should be judicially selected (mostly
the rarely used ways of writing code) so that they only affect a tiny
subset of existing programs. Such migration should also be slowly done
in a period of 5 or 10 years.

> Rejecting the
> redirection with space "3>& 1" or "3>& -" under the reason that you
> happened to know it only recently is something like requesting C
> language to change to prohibit « a=b » by the reason that you recently
> know the way of writing and most people and tutorials don't do that
> way.

The real question is whether a feature is commonly used or. It is a
feature few people use, then there is no reason to keep it.

One way to evaluate how many people are potentially affected. It can
be done to evaluate all the open-source code (e.g., on github) to see
how likely things like "3>& 1" is used. If it is rarely used, then it
could be a candidate for removal.

As long as a decision is based on real data, I don't see a problem
with such a rational decision.

> It's a strange request from the viewpoint of those who get used
> to the syntax. Maybe you can design such a new shell,

I am not asking to change the current bash code.

> but I'm not sure
> if it's worthwhile to change the existing behavior taking the risk to
> confuse the people who know that "3>& -" is possible in traditional
> POSIX shells. Actually, they are not that surprising for people who
> use « exec {fd}> xxx » and « echo >&$fd »

For the above piece of code, fd will not be `-`. I would require `-`
in `>&-` must be written explicitly as `-`'s really means closing. So,
I wouldn't allow something like the following to work.

i=-; exec 3>&$i

If people want to close fd 3, I'd require them to write it as `exec
3>&-`. Once your always require people to write >&-, then the question
becomes whether it is better to distinguish it as two parts ">&"
meaning direction, "-" meaning closing, or just think it a single
entity ">&-" closing a file descriptor. To me, the 2nd way of
understanding is more intuitive.

> or « exec 4>&$fd- », in
> which the file descriptor and - are clearly not the part of the
> redirection operator.

Bash can know a problem for this case.

$ i=; exec 3>&${i}-
-bash: ${i}: ambiguous redirect

So I think bash could have given an error for this as well. I can
understand when people is not sure what a destination file descriptor
is. It just doesn't make sense that a program doesn't know whether an
fd should be closed or not and have to assign it to a variable and let
the variable tell whether the fd should be closed or redirected.

i=-; exec 3>&$i

> > In that sense, if the goal is not to be POSIX compatible, but to have
> > something as compatible as to existing bash, and getting off those bad
> > features that most people don't use anyway, it would be helpful to
> > analyze the existing syntax. I hope that you understand the logic
> > reasoning here.
>
> You are talking about the goal of this specific discussion but not the
> goal of Bash, right?

Yes. Any actual changes of Bash implement at this level (which
potentially are used by many people unless there is real code analysis
like I mentioned for analyzing open-source code proving the opposite)
is too late.

-- 
Regards,
Peng