Re: updating shopt compat settings to include current version

[Mike Frysinger]

On 15 Oct 2015 16:06, Chet Ramey wrote:
> On 10/15/15 3:27 PM, Mike Frysinger wrote:
> > our build environment relies heavily on bash.  in our ebuild standard, we
> > declare the min version of bash that is supported (3.2 currently).  this
> > way we don't have people using features found only in newer versions and
> > breaking on systems with older versions of bash.
> 
> Wow.  How many systems have bash-3.2?  It is ten years old, after all.  Do
> you build and support new packages for old systems running bash-3.2?

we support it -- if someone reports it.  how many reports we actually get
who knows.  in practice, it means we disallow newer features like ${foo^^}
and ${foo,,} that are easy to recognize, and we try to write code like 
"${PV/_/'~'}" that'll work in both new and old versions.

we're in the process of picking a newer version for the next standard.  it'll
be 4.2 or 4.3.  but discussion for making upgrades less flaky shook out of the
discussion hence we looked at this compat logic.

> > but when bash changes behavior on us (there are a variety of examples 
> > between 3.2 and 4.3), some
> > ebuilds break because they expected bash 3.2 behavior but got something
> > else.  so every bash release ends up triggering a fire drill where we try
> > to weed out all the common issues (we have ~20k packages, and many have
> > multiple versions).
> 
> Do you ever do any of this work with the alpha and beta versions?  It
> seems like there are months there when you could be addressing these
> issues.

yes, when you release alpha/beta versions ;), i add them to Gentoo.  but devs
have to explicitly opt into them.  i do, and i know one or other people do, but
otherwise i'm not sure how much testing they see.

i've reported at least one such issue already, and i'm pretty sure we've seen
some in the past that got fixed before the final.  but the alpha/beta business
is still somewhat new.

> >  even then, for a while people introduce bugs since
> > they're running the new version only (or old version), and people notice
> > because things break.  then it settles down once everyone has moved to
> > the same version.  but i'm sure even today we add code that does not work
> > under bash-3.2 (or some version between then and now), but most people do
> > not notice, and those who do aren't guaranteed to file bugs.
> 
> You could always just run bash-3.2 against your ebuild system, or run the
> collection of scripts under bash-3.2, to test whether or not it works.

that assumes that behavior changes only once between versions.  pretty sure
we've seen changes where bash-3.2 did one thing, bash-4.3 did something else,
and versions in between did yet another thing.  i don't recall exact examples
off the top of my head, but i *feel* like it has happened :).

> > the bash compat feature seems to address this nicely: our standard says
> > we should use bash-3.2, so we set the compat level to that, and then we
> > have much stronger confidence in newer versions not breaking, or people
> > adding code that only works on newer versions.
> 
> You should approach this with caution.  I'm sure there are changes and bug
> fixes that introduce incompatible behavior that are not addressed by the
> compatNN variables.  I guess as long as it doesn't bite you, you're ok.

right.  we're OK with reporting & getting those fixed.

> > at any rate, it looks like the intention is to not do what we want, so
> > we'll have to explicitly check the BASH_VERSINFO ourselves up front and
> > deal with the various changes in compat selection.
> 
> Is it just the ebuild system or do you want to make sure every shell sets
> compat32?

only the ebuilds.  we explicitly do not want external scripts getting screwed
up.  that's even more unmanageable :).

> I mean, in theory, it's simple to do that:
> 
> unset BASH_COMPAT
> shopt -u compat31
> command shopt -s compat32 2>/dev/null

... but that doesn't work in bash-3.2:
$ bash-3.2 -c 'shopt -s compat32'
bash-3.2: line 0: shopt: compat32: invalid shell option name

and it won't work when we updated to bash-4.2/4.3, or when you stop adding
new compatXY options.
-mike

signature.asc
Description: Digital signature