Re: [sr #108491] Is `set -o histexpand` meant to trump `set -o posix`? [eg, echo "#!/"] (Because it does.)

[Chet Ramey]

On 2/1/14, 7:35 PM, Geoff Nixon wrote:
> So having done some more playing around with this, I've found that when
> bash is configured and compiled with '--enable-strict-posix-default', this
> doesn't occur.  I see this is documented in the change log for 4.3alpha, as:
> 
>   k.  When compiled for strict Posix conformance, history expansion is
> disabled by default.
> 
> But why is this a compile-time-only behavior?

It's a compromise between one of the oldest bash features -- history
expansion predates posix mode by a lot of years -- and the desire to
provide users a way to get a strictly-conforming Posix environment if they
really want one.

I got several requests to disable it entirely by default in bash-4.3, but
that's going to have to wait for a future version.

> There's nothing else in the documentation about any differences between
> bash compiled with -DSTRICT_POSIX v.s. enabling this mode with `bash
> --posix`. It just says that POSIX mode will become a default option. 

It doesn't say that exactly.  What the configure help string says is that
it will force bash to be conformant by default.  Even when you enable
Posix mode, the behavior controlled by xpg_echo and histexpand has to be
tweaked to get a really strictly conforming environment.  Since bash enters
Posix mode when it's called as `sh', I judged it would break too many
scripts to do away with `echo -n' and too many interactive user
expectations to disable history expansion.  So, posix mode and history
expansion (and echo behavior) are decoupled.

> Bash Version: 4.3
> Patch Level: 0
> Release Status: rc2
> 
> Description:
> 
> Please see these:
> 
>  
> http://stackoverflow.com/questions/21494213/why-cant-i-echo-the-characters-in-a-string-in-bash
> 
> http://stackoverflow.com/questions/21496049/what-is-the-intended-effective-ordering-of-set-o-options-in-bash-does-histe
> 
> 
> Its still not at all clear to me if I'm correct or not; and I kind of got a
> serious beatdown for calling it a bug on Stack Overflow. So I'm not sure if
> this is a 'blocker' per se or even something you'll consider a bug. But it
> certainly blocks me from wanting to using bash to execute a POSIX-portable
> script!

See above for the details about why the two expansions are decoupled, but
I'm not sure why you would have history expansion enabled when executing a
script.  It's only active in interactive shells.  I see what you're trying
to do to create a script, but when I put those commands in a script and run
it, I don't see history expansion attempted.

> (Note that I now fully realize my statement there to the effect of 'bash never
> sees any updates so use a different shell' is erroneous. I made this claim
> because a keep a git checkout on my hard drive which gets pulled daily - but
> your master branch is configured to track 4.2 rather than devel, so that's
> what I thought was indeed the case. I was completely aghast to find that 4.3
> is in rc2 having just made that statement, and I intend to redact it a soon as
> there's some resolution to this "bug", be it a bug or not.)

Bash-4.3 has been in testing for almost a year (4.3-alpha came out in
March, 2013), and public test versions have been available for six months.
I got hammered a little bit about the bash-4.2 testing period.

The bash git tree had been set up to track releases on the `master' branch
before I started using it heavily, so I kept it that way and added the
`devel' and `bash-4.3-testing' branches.

> I apologize if this appears in your inbox two dozen times. I don't know if
> sendmail even works at all on my system; and there are three slightly
> different emails in the bashbug script, in git, and the top hit in a web
> search for your name. I'm trying to make sure this gets seen before the
> 'release' button gets pushed for 4.3.

They all work to get to me.

> 
> Repeat-By:
>     set -o histexpand && set -o posix && echo "#!/"
>        set -o histexpand && set -o posix && echo "#!/bin/bash"
>         echo $?

This is still going to be the case in bash-4.3.  Only single quotes and
the backslash inhibit history expansion.  This is always been the case,
and I don't think it's worth it to change that now.  I would rather just
disable it entirely by default.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/