Re: Changing the way bash expands associative array subscripts

[Chet Ramey]

On 4/13/21 11:11 PM, Koichi Murase wrote:
> 2021年4月14日(水) 0:24 Chet Ramey <chet.ramey@case.edu>:
> > On 4/13/21 5:01 AM, Koichi Murase wrote:
> > > But I expected some design consideration enabling a[$key] for an
> > > arbitrary key in the indirect expansions and namerefs.
> >
> > Why? Why should the shell carry around (and expect the user to remember)
> > information about the original value assigned to a variable pre-expansion?
>
> I didn't mean we should preserve the syntactic information, which I
> don't either think is a good idea.  Maybe we could always treat
> `assoc[@]' as a single element reference.

Do you mean, for example, in assignment statements? I can see it in that
case, and for  `unset', and maybe for `test', because of the semantics of
builtin commands.

But if you take it farther than that, then there is no way to get the
elements of an associative array without introducing yet more syntax. My
guess is there are many, many more cases where ${assoc[@]} is intended to
mean "all the array elements" instead of "the element '@'", and I'm not
interested in that kind of incompatibility.


> > > If you are talking about the new behavior as I expect, why would
> > > «iref='a[\@]'» work?
> >
> > Why would it not? The rationale for the change is to eliminate the double
> > expansion of array indices that happens under certain circumstances, not
> > eliminate expansion of array indices entirely.
>
> I think now I understand the proposed change.  So, indirect expansions
> and name references are not planned to be changed because it already
> undergoes only a single set of expansions at reference time?

That's the thinking, yes.


> > Under what circumstances should they not be expanded? Because the
> > indirect expansions of array subscripts still undergo a single set
> > of word expansions.
>
>  From the users' point of view, indirect expansions and name references
> currently undergo "double expansions" in assigning time and in
> reference time; I mean naive users will write as « iref=a[$key] »
> instead of « iref='a[$key]' » and run « echo "${!iref}" » to find that
> the original right-hand side of the assignment is finally
> double-expanded until the reference time.

That's the point of indirect expansions! Even in the most basic use case:

foo=bar
bar=qux
v=$foo
echo ${!v}

nobody should be surprised to see a `double expansion'. I suppose you
can't protect people from their bad assumptions, but for the life of me I
don't see why anyone would not expect

key=one
iref=a[$key]
echo $iref

to display `a[one]', and it's completely straightforward to explain that
${!iref} and ${a[one]} have the same behavior.


> But, at the same time, I think it is also a valid discussion that we
> want to keep « ${!iref} » and « eval "\${$iref}" » equivalent to each
> other.  There is no perfect solution.  If one takes a rational
> semantics, I think the current behavior of ${!iref} should be kept.

I think I'd come out in favor of rational semantics.

> If one takes the semantics that naive users won't be surprised, I
> think the subscripts shouldn't be expanded at the reference time.  In
> any way, I would like to see consistency with « ref=a[$key]; unset
> "$ref" ».  If one takes the rational semantics, I believe the `unset'
> builtin should expand the array subscripts by itself so that « unset
> 'a[$key]' » works (as with the current default, shopt -u
> assoc_expand_once).  If one takes the semantics that is friendly to
> naive users, the `unset' builtin can stop its own expansion (shopt -s
> assoc_expand_once).

Don't limit yourself to `unset'. The same thing happens with test and
other builtin commands because of the set of word expansions they undergo
before the builtin even sees its arguments.

> I actually agree with konsolebox that assoc_expand_once for unset
> shouldn't be defaulted.  The option `assoc_expand_once' is incomplete
> in the sense that the behavior of `a[@]' and `a[*]' are subtle.  I see
> the current default behavior (with `assoc_expand_once' turned off)
> more consistent and clean. 

Yeah, maybe. But explaining the requirements for quoting things in multiple
ways is confusing, even to experienced users, and leads to knee-jerk
overreactions like "don't use associative arrays ever" that don't help
anyone. That's one of the motivations for this entire discussion.

 It seems to me that the option
> `assoc_expand_once' is just for naive users who fail to properly quote > the 
> arguments of `unset' but is inconsistent and ambiguous.

No. The `assoc_expand_once' option wasn't intended for `unset' at all. It
was to solve the double-expansion problem in other array contexts, like
arithmetic, as I said in my original message.

> and we can tell users to always write « unset 'a[$key]' », «
> iref='a[$key]'; echo "${!iref}" » and « declare -n nref='a[$key]';
> echo "$nref" ».

That's what I thought people would do with arrays in the first place. It
obviously hasn't worked out the way I anticipated.

So the current state has people doing

declare -A assoc
key='x]'

assoc[$key]=hello
declare -p assoc

unset assoc["$key"]
declare -p assoc

and it just plain doesn't work, even with assoc_expand_once.


> I completely agree on the change on the arithmetic expression ((
> assoc[$key] )).  I don't like the behavior of `unset' under
> `assoc_expand_once' because it seems be inconsistent to me, but there
> may be different perspective.  But, If `assoc_expand_once' is needed
> to make the behavior more friendly to naive users, I think we should
> also take care of naive users who write « iref='a[$key]'; echo
> ${!iref} ».

`assoc_expand_once' wasn't intended to "make the behavior more friendly
to naive users." It was intended to make the behavior safer and avoid
unintended word expansions that possibly included command substitutions.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/