Re: Bash-5.2-rc3 available

[Kerin Millar]

On Mon, 29 Aug 2022 09:53:39 -0400
Chet Ramey <chet.ramey@case.edu> wrote:

> On 8/26/22 9:29 PM, Kerin Millar wrote:
> > Hi Chet,
> > 
> > On Fri, 26 Aug 2022 14:28:00 -0400
> > Chet Ramey <chet.ramey@case.edu> wrote:
> > 
> >> 1. Changes to Bash
> >>
> >> a. Added a compatibility mode feature that causes the parser to parse 
> >> command
> >>     substitutions as if extglob were enabled. If it is enabled before 
> >> execution,
> >>     parse at execution will succeed. If not, the subsequent execution 
> >> parse will
> >>     fail.
> > 
> > I harbour some concerns as to both the utility and robustness of this.
> 
> Thanks for the report. This was an issue with both constructs using the
> same mechanism for parser error recovery and colliding, an easy fix.

Good! That's certainly a relief.

> 
> 
> > For extglob to be arbitrarily enabled in either context is unexpected (and 
> > undesirable). 
> 
> The conditional command behavior is compatible with ksh93, and has been
> this way for nearly twenty years. It's documented to behave that way. One
> would think people are used to it by now.

I am aware of how the == and != operators work, and that the pattern on the 
right hand side is always treated as an extglob in the shell performing the 
test [*]. I don't see what bearing that has on my report or its wording. Might 
I enquire as to what it is that I am supposed to be used to on account of the 
passage of those twenty years? Certainly not for a command substitution on the 
right hand-side of the == operator to implicitly perform the equivalent of 
shopt -s extglob within the implied subshell without so much as a by-your-leave 
on the part of the programmer, much less doing so in the shell that specified 
the command substitution. To make that observation was neither contentious, nor 
indicative of a need to study the manual or the CHANGES file more astutely.

For the benefit of anyone else reading, the following amounts to an error in 
bash 5.1.16, as it should.

$ shopt extglob
extglob         off
$ [[ '' == $( : !() ) ]]
bash: command substitution: line 3: syntax error near unexpected token `('
bash: command substitution: line 3: ` : !() )'

> 
> The parser change to enable this for backwards compatibility is controlled
> by the compatibility mode, and requires an explicit action to enable. It's
> a choice to enable compatibility mode, and choices have consequences;
> compatibility mode is intended to provide an opportunity to update scripts,
> not provide perfect emulation of a previous version.

I understand that you are striving to do right by a majority of your users at 
all times. The reported issue was a consequence of a regression in a late-stage 
release candidate introduced by a new backward compatibility feature that I had 
not elected to enable. I found this regression somewhat startling. I hope that 
you can understand where my concern stems from, at least.

> 
> I understand from Sam James that Gentoo globally forces the compatibility
> mode to 5.0. That, too, is a choice.

Yes, that's right - subject to the "EAPI" level defined by a given ebuild. I 
would aver that all of this amounted to a storm in a teacup. In only one single 
instance was a pathname expansion found that was incorrectly attempting to 
enable the extglob option after parsing has occurred, as was learned in the 
course of Sam and I proceeding to work together on this issue. I must emphasise 
that it's not that I don't think that backward compatibility is important. Far 
from it. Yet, I can't shake the feeling I had from the outset that, in this 
particular instance, the compromise may simply not be worth it. I considered 
the behaviour of the prior release candidates to be a step in the right 
direction. In any case, I have already expressed as much so I'll leave it at 
that and shall look forward to the next release candidate, should there be one.

> 
> > Paradoxically, this breaks one of the QA tests implemented by portage.
> 
> It's certainly ironic.

In case you were wondering, it's a test that entails stashing the output of 
shopt then later comparing, so as to determine whether an ebuild/eclass changed 
any options without restoring their prior values. Coercing extglob on results 
in a constant stream of false positives.

> 
> -- 
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>                ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/
> 

[*] Notwithstanding the change introduced by 4.1, whereby it was no longer 
necessary to either enable extglob first or store the pattern in a variable 
that was subsequently expanded on the right hand side of the operator.

-- 
Kerin Millar