Re: Unset array doesn't work

[Koichi Murase]

Hi everyone,

I'm not a member of Bash developers, but I would like to leave
comments as one of Bash users.

----------

I don't agree with changing the default behavior that removes the
placeholder of previous-context variables. The reason is just backward
compatibility, but it's important. For example, there is actually a
widely-used technique, "local $1 && upvar $1", relying on the behavior
of "unset" that removes the placeholder of the previous-context
variable. Everyone who has even once looked inside "bash-completion"
implementation should know this because bash-completion framework
extensively uses this technique. Usually "bash-completion" is
defaultly loaded in major Linux distributions, and changing the
default behavior will break the completion in such wide environment.
Also the change will break all the other scripts that uses "local &&
upvar" technique. If "unset" doesn't remove placeholders, there is no
alternative ways to do the similar thing unless some new builtin
"unlocal" or something is introduced. For another example, the
behavior of "unset" can be used to get the value of hidden global
variables, leaving the hiding local variables, by repeatedly calling
"unset" in a subshell. As far as I know, there is no other way to get
the value of hidden global variables although the value can be easily
set by "declare -g". In fact I use both techniques in my scripts. In
addition I won't be surprised if there is other useful techniques that
uses the current behavior of "unset".

----------

I agree with that the current context-dependent behavior of "unset" is
the source of confusion as I happened to know this behavior just two
years ago. At least, I believe, this context-dependent behavior should
be written somewhere in the documents. If a new shell option that
resolves the context dependence will be introduced, I prefer to have a
switch that changes the behavior of the local-scope "unset" but not
the previous-scope "unset". The reason is the current effects of
local-scope "unset" can be satisfied by bash's "local" builtin while
previous-scope "unset" have no alternatives. For example, the example
provided by kre

>        myfunc() {
>                local IFS
>                unset IFS
>                # do some code
>        }

is actually equivalent to the following code:

>        myfunc() {
>                local IFS
>                # do some code
>        }

If one wants to make an existing local variable have unset attribute
in the local-scope placeholder, one can write as

> func() {
>     local a=1
>     # do something
>     unset a
>
>     local a
>     # do something
> }

which should work for both side of the option that switches the
local-scope "unset".

----------

I don't think it's a bug that "unset" removes the placeholder of the
previous-context variables. It's just an intentional "design" as is
known from Chet and other people's replies. And also I don't see any
strong enough reason to conclude that it's a kind of a "design bug".
In fact the behavior of shells varies for "unset":


Example 1 (unsets previous-context variables):

Note: Some shell supports local variables with the "typeset" builtin.
For such a shell please replace "local" by "typeset" in the following
examples.

$ echo $(a=1; u() { unset a; }; f() { local a=2; echo a=$a; u; echo
a=$a; }; f; echo a=$a)

a=2 a= a=1 # zsh, dash, posh
a=2 a=1 a=1 # mksh, yash, bash


Example 2 (unsets variables defined in the local scope):

$ echo $(a=1; f() { local a=2; echo a=$a; unset a; echo a=$a; }; f; echo a=$a)

a=2 a= a=1 # zsh, dash, posh, bash
a=2 a=1 a=1 # mksh, yash


Versions of shells that I tested with:

- bash 4.4.12
- zsh 5.2
- mksh @(#)MIRBSD KSH R54 2016/11/11
- posh 0.12.6
- yash 2.46
- dash 0.5.9
- Note: ksh seems not support local variables. "typeset" in function
scopes defines a global variable, and "unset" removes the global
variable.
- Note: "busybox sh" is a variant of dash, so the behavior is the same.


These results means there is no single answer for the "correct design"
of unset. The only point that bash is confusing is its
context-dependent behavior.

----------

> var=set
> func() { local var; echo ${var-unset}; }
> echo $var ; func; echo $var

For this, again the result varies from shell to shell, so I think
there is no single answer for the "correct design" of "local".

Example 3:

$ echo $(a=set; f() { typeset a; echo ${a-unset}; }; echo $a; f; echo $a)

set set set # dash
set set # zsh
set unset set # mksh, bash, posh, yash

----------

Best regards,
Koichi

2018-03-03 16:23 GMT+09:00 Robert Elz <kre@munnari.oz.au>:
>     Date:        Fri, 2 Mar 2018 14:43:02 -0500
>     From:        Chet Ramey <chet.ramey@case.edu>
>     Message-ID:  <4502a0e5-0600-d294-9af2-4e9eeb0a058e@case.edu>
>
> My final comments on this subject:
>
>   | Perhaps. But bash has never done this. Not from day one. That's 30 years.
>
> That a bug (be it a design bug, or a coding bug) has existed a long tiime
> does not make it any less a bug.
>
> I have been using bash for essentially all that time (from before you took
> over maintainership) and I never knew it worked like that.   From comments
> here (where some people far more knowledgable about bash and its
> internals than I are to be found) I suspect that very few other people know
> about it either.
>
>   | This is how bash dynamic scoping works. The exception for the declaration/
>   | unset at the current scope was added 16 years ago, and the existing
>   | behavior was already entrenched.
>
> And yet when that change to the entrenched behaviour was made,
> there were no complaints?   And there's no option to switch back to
> the previous way?   Kind of suggests just how important everyone
> believes the original method was, doesn't it?
>
>   | I can see doing this and allowing it to be toggled by a shell option.
>
> A suggestion:   Do that for bash 5, and in the alpha release, make
> the option default to cause things to work the opposite way than
> happens now (so the option needs to be explicitly changed to get
> the current behaviour).   I know that's the opposite of what would
> usually be done in order to retain backwards compat, but for this,
> I think it would be a useful test to see if anyone notices the difference.
> You can always change it for beta/final releases if there are issues.
> If not, perhaps the option can just go away (then or later.)
>
>   | > Lastly, where does the notion of "remove" come from?
>   |
>   | As a way to describe the historical bash behavior, it works.
>
> Yes, that I understand.   My issue is that I believe this is colouring
> your thoughts on just what "unset" is - same as the "appear/be"
> (trivial seeming) semantic issue you commented on in another message.
>
> That is, it appears to me as if you believe that "unset" (as a state, not
> the command here) implies "non-existing".   That's never been correct.
>
> The converse is correct - a variable that does not exist appears as
> an unset variable when referenced.
>
> There are (even ignoring the unset command) too many ways
> (in bash, as well as other shells) to get variables that patently
> obviously "exist" in some form or other but are unset.
>
> The most obvious example is
>
>         export newvar
>
> after that
>
>         echo ${newvar-unset}
>
> prints "unset".   Sometime later if we give newvar a value, it, and its
> new value are exported - demonstrating that the export attribute was
> remembered (ie: "newvar" existed before it was set - it must have done
> in order to retain an atttribute).
>
> jinx$ export newvar
> jinx$ echo ${newvar-unset}
> unset
> jinx$ newvar=set
> jinx$ printenv newvar
> set
> jinx$ echo $BASH_VERSION
> 4.4.12(1)-release
>
> All shells that function correctly behave that way.   In bash (and in
> several other shells, though not all, due to unrelated differences)
> the same is true of the local command in a function...
>
> var=set
> func() { local var; echo ${var-unset}; }
> echo $var ; func; echo $var
>
> prints "set" "unset" "set"  showing that in the function there is
> a var (which must still exist as its value is retained in the global
> scope) which is unset.
>
> Even if the model was that the "var" in the function is something
> completely unrelated to the global "var" (as it would be in C for
> example) we can still see that the local command must have
> created some state for it, as otherwise the echo would be
> referring to the global "var".
>
> This can be more obviously seen in
>
> var=set
> func() { [ -n "$1" ] && local var; echo ${var-unset}; }
> echo $var ; func x; func; echo $var
>
> The same code either accesses the local or global var
> depending upon whether we happened to have executed
> the local command or not - indicating that command sets
> some atttibute, and still makes an unset var.
>
> Again this is all as it should be (given the assumption that "local"
> creates an unset version of the variable, which is a rational choice,
> if not the one made by all shells - a topic we have discussed in
> another context.)
>
> The best model for all of this is that set/unset is an atttibute of a
> shell variable, just like exported, read-only (and any more that
> exist in various shells) and that when we reference a variable
> which does not (or perhaps even just did not) exist at all, then
> it is made to appear as if it were an unset variable, and this is
> the special case, rather than considering leaving some state
> around to remember attributes of vars that happen to be unset
> but must continue to exist as being what is unusual or special.
>
> An analogy is the unlink sys call, which is not "remove".   If you
> think of it as remove, and that its function is to remove files, then
> things get way too complex to describe the behaviour.   But if
> it just alters an attribute of the file (in that case, its link/ref count)
> it all becomes simple.   And then when there is no longer any
> way to access the file, it can be removed as a space saving
> optimisation.     Treat the unset command in the shell the same
> way.  If, after it, there is nothing about the variable that is different
> than a variable which does not exist, then we can remove it
> because that changes nothing, and saves memory  - but if there
> remains anything about the variable which is still needed, then
> we just cause the unset atttibute to be set (however that is
> implemented) and continue.
>
> kre
>
>