Re: devel: Questions about quoting in the new replacement ${var/pat/&}

[Chet Ramey]

On 1/13/22 7:35 AM, Koichi Murase wrote:

> The current behavior is as follows:
>
> $ bash-dev --norc
> $ shopt | grep patsub
> patsub_replacement      on
> $ var=ABCDE value='(K&R)'
> $ echo "${var//C/$value}"
> AB(KCR)DE
> $ echo "${var//C/"$value"}"     # (My suggestion of "quoting &")
> AB(KCR)DE     # (the current patsub_replacement isn't designed to work for this)
> $ echo "${var//C/(K\\&R)}"
> AB(K&R)DE
> $ echo "${var//C/\\\\\\\\}"
> AB\\\\DE
> $ echo "${var//C/\\\\\\\\&}"
> AB\\CDE
>
> Today I tried to modify my script so that it works with `shopt -s
> patsub_replacement', and now I become more skeptical about making
> patsub_replacement default and also about its current design.

Maybe I've mellowed with age. You'll see what I mean.

> I needed to modify more places than I initially expected, and also it
> is not simple to simply perform the replacement that contains
> arbitrary strings. Then, I checked other existing Bash programs and
> found that most of the large Bash programs/frameworks are affected,
> which include "bashdb", "bash-completion", "bash-it",
> "bash-oo-framework", "neofetch", and "bashtop" (plus mine, "ble.sh").
> The programs that seemed to be fine among the ones I have checked were
> only "oh-my-bash", "romkatv/gitstatus" and "git-prompt.bash". So I
> feel it is better to make patsub_replacement off by default. 

I generally make new functionality that's controlled by a separate option
off by default in releases. I'll do the same here. It's on now so we can
evaluate its effects. It's hard to get enough people to test alpha and
beta releases, so who knows what good that will do.

> Sorry, but I'd like to still push treating the quoted replacement (as
> $string in « result=${result// /$string} ») literally just like the
> glob operators in ${var//"$pat"} or ${var#"$pat"} (as recently
> explained in e.g.
> https://lists.gnu.org/archive/html/help-bash/2022-01/msg00022.html).

They're not patterns, but I'm coming around to this viewpoint. They do
undergo some kind of interpretation.

It probably won't be in bash-5.2-alpha, since I already froze and tested
that, but the behavior will be different in the next devel branch push
after that.

> > > > * It is consistent with the treatment of the glob special characters
> > > >    and anchors # and % in $pat of ${var/$pat}.
> > >
> > > Yeah, doing that was probably a mistake, but we have to live with it now.
> > > Those are really part of the pattern operator itself, not properties of
> > > the pattern. But nevertheless.
>
> but I don't feel it is a mistake but rather natural than introducing
> an extra level of unescaping/unquoting. It might be just because I got
> used to it, but

They're operators. But like I said, it's too late now.

> > However, if I understand it correctly, similar treatment is
> > already standardized in POSIX for ``quoting characters within the
> > braces'' (as ${var#"$xxx"} and ${var%"$xxx"}):
> >
> > > quoted from POSIX XCU 2.6.2
> > > https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/utilities/V3_chap02.html#tag_18_06_02
> > > [...]
>
> which means most shell implementations agreed with its behavior for
> glob operators in ${var#pat} and ${var%pat}.

Right, and that's part of the argument for allowing quoting to inhibit the
replacement in pattern substitution. They're not patterns, exactly, but
they do contain characters that have a special meaning.

I still like the simpler explanation: if bash-5.1 expanded `string' to
something that contained an unescaped `&', replace it. If it expanded to
something that contained a literal '\&' you'll get a `&'. Backslash-escaped
backslashes produce a single literal backslash. But allowing double quotes
to inhibit the replacement is probably better for backwards compatibility,
even with patsub_replacement off by default.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/