Re: unset does not act as expected on namerefs

[Greg Wooledge]

On Tue, May 26, 2015 at 05:02:43PM -0400, Shawn Wilson wrote:
> If there's no good reason to keep this as is (some use case where
> this might be wanted and breaking backward compatibility - I can't
> see anyone actually *wanting* it this way) shouldn't it be changed?
> A behavior can be documented and still be bad.

Well, there are multiple aspects to this issue.

The first is that "declare -n" is very new (only since bash 4.2), so
most script writers either aren't aware of it yet, or haven't had
much chance to use it.  It's hard to have a good understanding of
how to use something properly without some experience.

The second is that bash's "declare -n" is, in its current implementation,
not robust enough for general use.  It has two major issues, which I've
documented on my wiki, which I'll summarize here:

 1) It doesn't cross scopes.  It's not like Tcl's upvar at all.  It
    only refers to a variable in the same scope (which, following the
    standard bash rules, means it'll recursively search upward until
    it finds a matching variable by name).  What this means is that
    while you might EXPECT this to work:

    f() {
        declare -n foo="$1"
        foo=set_by_f
    }

    It won't work in the general case.  Demonstration:

    imadev:~$ f() { declare -n foo="$1"; }
    imadev:~$ g() { declare -n foo="$1"; f foo; }
    imadev:~$ foo=bar
    imadev:~$ g foo
    bash: warning: foo: circular name reference

    So, you can't use it to pass variable names from a caller to
    a function.  You STILL have namespace collisions.

 2) It allows arbitrary code execution, just like eval:

    imadev:~$ f() { declare -n foo="$1"; echo "$foo"; }
    imadev:~$ f 'x[i=0$(date >&2)]'
    Wed May 27 08:07:35 EDT 2015


    Though, one might argue that this is more of an issue with bash's
    indexed arrays than with eval or declare -n.  Even printf -v isn't
    safe against this.

(See also http://mywiki.wooledge.org/BashFAQ/048)

The third aspect to consider about "unset nameref" is whether the script
writer's intent was to unset the nameref itself, or the variable pointed
to by the nameref.

    foo=bar
    unset foo

Given those two lines of code, with no context, we might expect that
the word "bar" is no longer held in memory anywhere (or is held in a
chunk of memory that's free to be overwritten at any time).  If foo
is a regular variable, this is the case.  If foo is a nameref to
another variable, then it's STILL true:

    imadev:~$ declare -n foo=somevariable
    imadev:~$ foo=bar
    imadev:~$ unset foo
    imadev:~$ declare -p foo somevariable
    declare -n foo="somevariable"
    bash: declare: somevariable: not found

This leaves the nameref intact, so you can assign a new "somevariable"
through it.  Wiping and re-assigning the pointed-to variable might be
what someone wants.

Here's another surprise, that I didn't know until now.  Given the above,
if we follow it up with another declaration of foo, it "hides" the
nameref.   But the nameref declaration is still there, lurking, waiting.

    imadev:~$ declare -A foo
    imadev:~$ foo["jug"]="brown"
    imadev:~$ declare -p foo somevariable
    declare -A foo='([jug]="brown" )'
    bash: declare: somevariable: not found
    imadev:~$ unset foo
    imadev:~$ declare -p foo somevariable
    declare -n foo="somevariable"
    bash: declare: somevariable: not found

Is that what Chet intended?  I have no idea.  (I was actually expecting
"declare -A foo" to create an associative array named somevariable,
with foo still pointing to it.)