[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Incorrect alias expansion within command substitution

From: Chet Ramey
Subject: Re: Incorrect alias expansion within command substitution
Date: Wed, 2 Feb 2022 11:38:30 -0500
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.5.1

On 2/1/22 5:31 PM, Robert Elz wrote:
>     Date:        Tue, 1 Feb 2022 15:39:06 -0500
>     From:        Chet Ramey <chet.ramey@case.edu>
>     Message-ID:  <2816cf78-d7be-b9e1-733d-12427b04cb49@case.edu>
>   | When you say "just parsed," when are aliases expanded?
> During lexical analysis, right between when the input is read (if it
> is read, and isn't from some internal string) and when it is handed to
> the grammar.
>   | Are they expanded while scanning the command substitution to find the
>   | closing `)' but not part of the text that results?
> For the former half, yes.   And I think you're aware that we have no
> "text that results" - which is why, among other things, <<$(anything)
> doesn't work for us, as the $(anything) is a (kind of) a pointer to
> a piece of parse tree, which never ends up matching anything even remotely
> ascii.

I recall that discussion now.

> Keeping the original text is something on my todo list, but it is complicated
> by our memory management methods - I could just malloc(biggish) and then
> realloc(2 * biggish) if biggish isn't enough, but I really would prefer not
> to do that if I can avoid it (this is a temporary string, it doesn't need
> semi-permanent storage, which is what malloc() is used for in our shell).

It was more intrusive than I wanted; producing reconstituted text from the
parse tree reused the xtrace code anyway, and doesn't really affect
anything semantically.

>   | but bash has not traditionally done it that way. That's where the 
> backwards
>   | compatibility issues come in.
> Oh.  I see.   Does anyone really care though?   It is kind of hard to imagine
> anyone being perverse enough to use an alias in a function anywhere, let
> alone depend upon it changing between executions of a function when the alias
> definition has been altered.   Anyone doing anything like that deserves to
> have their code break IMO.

Ha, you'd be surprised. Not necessarily because they intended to do it that
way, but because they did it and it worked. Now it has to work forever.

>   | I considered keeping both the original text and the parse tree from the
>   | parsed command substitution (well, a chain of them since you can have an
>   | arbitrary number of command substitutions in a word).
> Yes, that's what we do.
>   | It's difficult, given bash's internal structure, to preserve the
>   | original text
> Same for us, probably for different reasons though.
>   | -- as opposed to the reconstituted text --
> How accurately can you reconstitute?   That is, can you maintain the
> difference between $(a b) and $( a b ) for example ?   How about $(a  b) ?

Does it make a semantic difference? The only thing I can think of that
might mess it up is when you have something like

alias l='(' r=')'
echo $(l 4+4 r)

and you intend, for whatever perverse reason, to have this parsed as an
arithmetic expansion. I'm not sure that's worth supporting.

>   | Other shells (bosh, mksh) also recreate the text of a command
>   | substitution from the parsed commands.
> Interesting, though I am not surprised in a way.   Actually needing the
> command sub in textual form (aside from perhaps showing in output from
> jobs or something, though I doubt it is ever needed even there - and for
> that kind of thing, a good approximation is just fine in any case) is
> very very rare in sh - the end word of a here-doc redirection operator
> might be the only case.

You need to be able to reconstruct the text of arbitrary commands for
`set -x'. It's a very short step from that to rebuilding a command

>   |
>   | > It seems to me that the way you're doing it now would also break:
>   | > 
>   | >         alias x=y
>   | >         cat <<$(x)
>   | >         whatever
>   | >         $(x)
>   | This works in bash default mode because aliases aren't expanded while the
>   | command is parsed.
> I don't know whether I was in "default mode" or not (all I did was run 
> "bash-5.2" which is the name where I stored the development binary - which
> is still the last one I built while running tests, I haven't built the 
> 5.2alpha
> version yet, so what I am running is about a month and a half old I think)

You were in default mode, since you did not take the active step to run in
posix mode.

> But given that I simply ran it, and then typed (pasted into an xterm
> actually, but that makes no difference) the commands above, and it didn't
> work (it did in bash 5.1.16 or whatever the current released one is).

Ah, I interpreted that as being run from a shell script, which is how I
tested it.

>   | The delimiter ends up being `$(x)'.
> In the version I tested (5.2 development version) it ended up being $(y)

Yes, since aliases get expanded while reading the WORD that is the here-doc
delimiter. That's what we're talking about changing here. It works in
previous versions of bash because those versions don't expand aliases while
(ad-hoc) parsing the command substitutions at all. It will work in default
(non-posix) mode versions going forward because I'm going to err on the
side of backwards compatibility, at least for a while.

>   | Since you're required to check the line read for the terminating
>   | delimiter before doing anything else, the delimiter has to be $(x)
>   | to make it work.
> Yes, I'm aware of that.
>   | It works with ksh93 as well, but every other shell produces an error of
>   | some sort (including the NetBSD sh, unless you've changed something in the
>   | couple of months since I last built it).
> No, it definitely does not work in our shell.   That's what I said a little
> later in my message.  When we process the redirect, the word has (more or less
> in this case) just a pointer to a parse tree (or as you mentioned above, to
> the head of a chain of parse trees, in this case, a chain of length 1).

I recall having that conversation now.

> Of course in reality, none of this truly matters.   Only an idiot would
> ever actually do something like
>       alias x=y
>       cat <<$(x)
>       whatever
>       $(x)

I suppose. But it's good to have the discussion about command substitution
parsing and when alias expansion happens nevertheless.


``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

reply via email to

[Prev in Thread] Current Thread [Next in Thread]