Bash-5.2-beta available

[Chet Ramey]

The first beta release of bash-5.2 is now available with the URLs

ftp://ftp.cwru.edu/pub/bash/bash-5.2-beta.tar.gz
https://ftp.gnu.org/pub/gnu/bash/bash-5.2-beta.tar.gz

and from the bash-5.2-testing branch in the bash git repository
(http://git.savannah.gnu.org/cgit/bash.git/log/?h=bash-5.2-testing).
You can use

git clone --branch bash-5.2-testing git://git.savannah.gnu.org/bash.git

to clone the testing branch.

The CWRU FTP site works best if your client supports Extended Passive
(EPSV) mode.

This tar file includes the formatted documentation (you should be able to
generate updated versions yourself).

This release fixes several outstanding bugs in bash-5.1 and introduces
a number of new features. There are significant new features of note:

* The parser now calls itself recursively to parse command substitutions,
  and reconstitutes the text from the parsed command. This allows better
  syntax checking and catches errors much earlier;

* Bash-5.2 improves associative array subscript parsing with the goal of
  eliminating double evaluation for shell constructs such as arithmetic
  expansion and conditional commands, the `unset' builtin, and arguments
  to other builtin commands as long as assoc_expand_once is set;

* Associative arrays now allow assignment to and unsetting `@' and `*' keys,
  and the `unset' builtin treats `@' and `*' keys differently to allow this;

* There is an alternate array implementation, selectable at configuration
  time, that optimizes access speed over memory use (but can use considerably
  more memory for large sparse arrays);

* There is a configuration option to compile $"..." string translation out of
  the shell, and a runtime shell option to inhibit any additional expansion
  of translated strings;

* The new `varredir_close' shell option causes bash to automatically close
  file descriptors opened with {var}<fn and other styles of varassign
  redirection unless they're arguments to the `exec' builtin.

Readline has a new option that allows users to disable the active region
without disabling bracketed paste. The option has the same default value
as bracketed-paste, and enabling bracketed paste enables the active region,
but disabling the active region doesn't disable bracketed paste.

There are two new bindable readline variables, active-region-start-color
and active-region-end-color. The first sets the color used to display the
active region; the second turns it off. If set, these are used in place of
the escape sequences that enable and disable terminal standout mode.

There are several notable bug fixes. Bash no longer leaves stray FIFOs when
expanding redirections before running a command from the file system. There
are fixes for several memory leaks and errors found via fuzzing. There are
fixes for problems with the `declare' builtin and its siblings that caused
occasional out-of-order evaluation. As usual, there is a complete list of
bug fixes appended to this message.

There are a few incompatible changes between bash-5.1 and bash-5.2. The
changes to eliminate double associative array subscript expansion mean
that some multiply-quoted expansions will behave differently. Allowing
assignments to associative array keys `@' and `*' means changes to the
`unset' builtin to allow them to be unset. You can still unset an entire
array variable A using `unset A'. Calling `unset I[@]' or `unset I[*]' for
an indexed array I now removes all the elements without unsetting the
array (identical to `I=()'). I have tried to put all of these cases into
the bash-5.1 compatibility mode. The change to command substitution parsing
means that errors are caught at different times than bash-5.1 and earlier
versions.

`bashbug' may be used to report bugs with this version.  It will send
mail to chet.ramey@case.edu if the shell's `release status' is alpha or
beta.

As always, thanks for your help.

Chet

+========== CHANGES ==========+
This document details the changes between this version, bash-5.2-beta, and
the previous version, bash-5.2-alpha.

1. Changes to Bash

a. Fixed a problem with command-oriented history and multi-line commands that
   caused embedded blank lines to be run together.

b. Changed the way `&' is quoted when performing pattern substitution and
   `patsub_replacement' is enabled.

c. Fixed some integer overflows when expanding strings or reading the output
   of command substitution larger than 2GB.

d. `wait -p' without the `-n' option now does something useful if there are no
   jobs.

e. Fixed an issue with read timeouts in posix mode.

f. Changed here-document processing to process $'...' and $"..." only when they
   appear in the WORD portion of ${PARAM OP WORD} in the here-document body
   and the body is being expanded.

g. Changed alias expansion in command substitution to be posix-conformant
   (performed while initially parsing the command substitution) when in posix
   mode.

h. Bash optimizes away more forks in subshells.

i. Here-document construction now performs quote removal on the here-document
   delimiter only if it's marked as quoted, which prevents quote characters in
   command substitutions from being removed.

j. Prompt string expansion now gives invisible characters in the expansion of
   the \w, \W, and \s escape sequences a visible representation to avoid
   problems with redisplay.

k. Fixed a problem with SIGINT during the execution of a command bound with
   `bind -x' affecting the saved terminal settings.

l. Fixed an inconsistency with how $@ expands in a construct like ${@:+set}
   or ${array[@]:+set} in the presence of null positional parameters or
   array elements.

2. Changes to Readline

a. Prevent some display problems when running a command as the result of a
   trap or one bound using `bind -x' and the command generates output.

b. Fixed an issue with multi-line prompt strings that have one or more
   invisible characters at the end of a physical line.

c. Fixed an issue that caused a history line's undo list to be cleared when
   it should not have been.

3. New Features in Bash

a. There is a new bindable readline command name: `vi-edit-and-execute-command'.

4. New Features in Readline

a. Two new bindable string variables: active-region-start-color and
   active-region-end-color. The first sets the color used to display the
   active region; the second turns it off. If set, these are used in place
   of terminal standout mode.

b. New readline state (RL_STATE_EOF) and application-visible variable
   (rl_eof_found) to allow applications to detect when readline reads EOF
   before calling the deprep-terminal hook.

c. There is a new configuration option: --with-shared-termcap-library, which
   forces linking the shared readline library with the shared termcap (or
   curses/ncurses/termlib) library so applications don't have to do it.

------------------------------------------------------------------------------
This document details the changes between this version, bash-5.2-alpha, and
the previous version, bash-5.1-release.

1. Changes to Bash

a. Fixed a bug that assigned a value to the variable name supplied as an
   argument to `wait -p' when there were no jobs.

b. Fixed a bug that resulted in performing incorrect word expansion on the
   key/value pairs in a compound array assignment.

c. Fixed a bug that could put the child forked to run a command substitution
   into the wrong process group.

d. Fixed a problem that could cause the lastpipe option to work incorrectly if
   file descriptor 0 was closed.

e. Bash tries to suppress traps if a forked child receives a trapped signal
   before it has a chance to reset its signal handlers.

f. Fixed several memory leaks in compound array assignments.

g. Fixed a problem with performing an assignment with `+=' to an array element
   that was the value of a nameref.

h. Fixed a bug that could cause a nameref containing an array reference using
   `@' or `*' not to expand to multiple words.

i. Fixed a bug where extended glob functions could match `.' or `..' when it
   wasn't explicitly specified, even if dotglob was set.

j. Fixed a bug that caused non-interactive posix-mode shells not to exit on a
   variable assignment error while assigning into the temporary environment.

k. Fixed a bug that caused parsing errors if an alias contained a compound
   array assignment.

l. Fixed a couple of instances where bash checked syntax too aggressively when
   trying to determine how to add a partial command to command-oriented
   history.

m. Fixed a parser problem that caused it not to allow reserved words to follow
   the `((' and `[[' commands.

n. Fixed a bad offset calculation when using negative offsets to `history -d'.

o. Fixed an off-by-one error that caused a read past the end of a buffer when
   reading a multibyte character from the output of a command substitution.

p. Fixed a problem with a failed `exec' command not setting $? to the right
   value for an exit trap.

q. Fixed a problem that caused bash not to unlink FIFOs created as part of
   expanding redirections for an external command.

r. Fixed a bug that could cause aliases not to be expanded in case statements.

s. Fixed a bug that could cause word completion to attempt programmable
   completion for the target of a redirection, instead of filename completion.

t. Fixed a bug that could result in errors after rebinding a key sequence with
   `bind -x' multiple times.

u. Fixed a problem that could result in not quoting the result when performing
   command name completion with a glob pattern in the command name.

v. `mapfile' now uses fully-buffered reads in more cases, which should improve
   bulk read performance.

w. Fixed a bug that caused `wait -n' to not reset its internal state when
   interrupted by a signal, resulting in subsequent calls failing.

x. Fixed a bug with parsing numeric arguments to readline key sequences
   installed with `bind -x'.

y. Bash suppresses forking in several additional cases, including most uses
   of $(<file).

z. If there are multiple `!' tokens in a [[ command, toggle the invert state
   instead of setting it unconditionally.

aa. Fixed a bug where running `fc' on an empty history list would cause the
    shell to crash.

bb. Word completion now checks whether or not a quote character closes a
    quoted string begun on a previous line, so readline doesn't interpret the
    quote as starting a new quoted string.

cc. Fixed a typo that translated \UNNNNNNNN characters that were not valid in
    the current locale encoding as \uNNNNNNNN.

dd. Fixed an issue that could cause bash to print timing statistics for the
    wrong command when `set -e' causes a command to fail.

ee. Bash now runs the exit trap in function context if a failed command in
    the function causes the shell to exit.

ff. Some fixes to how subshells modify $SHLVL.

gg. Fixed a bug that caused `mapfile -t' not to remove the delimiter when the
    delimiter is > 128 in the current encoding.

hh. Fixed a problem that could cause the shell to attempt to free unallocated
    memory if an expansion error occurred.

ii. Fixed a bug in the bash malloc implementation of malloc_usable_size. Bash
    did not use it, but it could cause problems with library functions that
    did.

jj. If the `exec' builtin fails, and the shell does not exit, it restores
    trapped signals to their trapped state.

kk. Fixed a bug that could cause variable assignment arguments to `declare' to
    expand variables using attributes that the call to declare was turning off.

ll. Fixed a bug with LINENO and arithmetic for commands.

mm. Fixed a posix-mode bug that caused tildes not to be expanded after an
    unquoted colon on the right side of an assignment statement.

nn. Fixed a problem with `caller' and line numbers when executing an ERR trap.

oo. Fixed a problem that could make the value returned by ${a:=b} not be the
    final value assigned to a (e.g., if `a' has an attribute that modifies
    the value on assignment).

pp. Fixed a problem with saving multi-line here-documents to the history list
    where the here-document delimiter does not appear on the first line.

qq. Fixed a bug with using += to assign to dynamic variables like RANDOM.

rr. Fixed a bug that caused `set -n' to modify $? if set after execution had
    started.

ss. Builtins like printf/read/wait now behave more consistently when assigning
    arbitrary keys to associative arrays (like `]'. when appropriately quoted).

tt. Fixed a problem with here-document collection printing the wrong prompt
    when parsing a compound list in an interactive shell.

uu. Fixed a problem with quoting shell expansion characters (like `$') when
    they appear in a tab-completed word along with characters that do need
    quoting (e.g.. $HOME/VirtualBox VMs).

2. Changes to Readline

a. Fixed a problem with cleaning up active marks when using callback mode.

b. Fixed a problem with arithmetic comparison operators checking the version.

c. Fixed a problem that could cause readline not to build on systems without
   POSIX signal functions.

d. Fixed a bug that could cause readline to crash if the application removed
   the callback line handler before readline read all typeahead.

e. Added additional checks for read errors in the middle of readline commands.

f. Fixed a redisplay problem that occurred when switching from the digit-
   argument prompt `(arg: N)' back to the regular prompt and the regular
   prompt contained invisible characters.

g. Fixed a problem with restoring the prompt when aborting an incremental
   search.

h. Fix a problem with characters > 128 not being displayed correctly in certain
   single-byte encodings.

i. Fixed a problem with unix-filename-rubout that caused it to delete too much
   when applied to a pathname consisting only of one or more slashes.

j. Fixed a display problem that caused the prompt to be wrapped incorrectly if
   the screen changed dimensions during a call to readline() and the prompt
   became longer than the screen width.

k. Fixed a problem that caused the \r output by turning off bracketed paste
   to overwrite the line if terminal echo was disabled.

l. Fixed a bug that could cause colored-completion-prefix to not display if
   completion-prefix-display-length was set.

m. Fixed a problem with line wrapping prompts when a group of invisible
   characters runs to the right edge of the screen and the prompt extends
   longer then the screen width.

n. Fixed a couple problems that could cause rl_end to be set incorrectly by
   transpose-words.

3. New Features in Bash

a. The bash malloc returns memory that is aligned on 16-byte boundaries.

b. There is a new internal timer framework used for read builtin timeouts.

c. Rewrote the command substitution parsing code to call the parser recursively
   and rebuild the command string from the parsed command. This allows better
   syntax checking and catches errors much earlier.

d. The `ulimit' builtin now treats an operand remaining after all of the options
   and arguments are parsed as an argument to the last command specified by
   an option. This is for POSIX compatibility.

e. Here-document parsing now handles $'...' and $"..." quoting when reading the
   here-document body.

f. The `shell-expand-line' and `history-and-alias-expand-line' bindable readline
   commands now understand $'...' and $"..." quoting.

g. There is a new `spell-correct-word' bindable readline command to perform
   spelling correction on the current word.

h. The `unset' builtin now attempts to treat arguments as array subscripts
   without parsing or expanding the subscript, even when `assoc_expand_once'
   is not set.

i. There is a default value for $BASH_LOADABLES_PATH in config-top.h.

j. Associative array assignment and certain instances of referencing (e.g.,
   `test -v' now allow `@' and `*' to be used as keys.

k. Bash attempts to expand indexed array subscripts only once when executing
   shell constructs and word expansions.

l. The `unset' builtin allows a subscript of `@' or `*' to unset a key with
   that value for associative arrays instead of unsetting the entire array
   (which you can still do with `unset arrayname'). For indexed arrays, it
   removes all elements of the array without unsetting it (like `A=()').

m. Additional builtins (printf/test/read/wait) do a better job of not
   parsing array subscripts if array_expand_once is set.

n. New READLINE_ARGUMENT variable set to numeric argument for readline commands
   defined using `bind -x'.

o. The new `varredir_close' shell option causes bash to automatically close
   file descriptors opened with {var}<fn and other styles of varassign
   redirection unless they're arguments to the `exec' builtin.

p. The `$0' special parameter is now set to the name of the script when running
   any (non-interactive) startup files such as $BASH_ENV.

q. The `enable' builtin tries to load a loadable builtin using the default
   search path if `enable name' (without any options) attempts to enable a
   non-existent builtin.

r. The `printf' builtin has a new format specifier: %Q. This acts like %q but
   applies any specified precision to the original unquoted argument, then
   quotes and outputs the result.

s. The new `noexpand_translations' option controls whether or not the translated
   output of $"..." is single-quoted.

t. There is a new parameter transformation operator: @k. This is like @K, but
   expands the result to separate words after word splitting.

u. There is an alternate array implementation, selectable at `configure' time,
   that optimizes access speed over memory use (use the new configure
    --enable-alt-array-implementation option).

v. If an [N]<&WORD- or [N]>&WORD- redirection has WORD expand to the empty
   string, treat the redirection as [N]<&- or [N]>&- and close file descriptor
   N (default 0).

w. Invalid parameter transformation operators are now invalid word expansions,
   and so cause fatal errors in non-interactive shells.

x. New shell option: patsub_replacement. When enabled, a `&' in the replacement
   string of the pattern substitution expansion is replaced by the portion of
   the string that matched the pattern. Backslash will escape the `&' and
   insert a literal `&'.

y. `command -p' no longer looks in the hash table for the specified command.

z. The new `--enable-translatable-strings' option to `configure' allows $"..."
   support to be compiled in or out.

aa. The new `globskipdots' shell option forces pathname expansion never to
    return `.' or `..' unless explicitly matched.

bb. Array references using `@' and `*' that are the value of nameref variables
    (declare -n ref='v[@]' ; echo $ref) no longer cause the shell to exit if
    set -u is enabled and the array (v) is unset.

4. New Features in Readline

a. There is now an HS_HISTORY_VERSION containing the version number of the
   history library for applications to use.

b. History expansion better understands multiple history expansions that may
   contain strings that would ordinarily inhibit history expansion (e.g.,
   `abc!$!$').

c. There is a new framework for readline timeouts, including new public
   functions to set timeouts and query how much time is remaining before a
   timeout hits, and a hook function that can trigger when readline times
   out. There is a new state value to indicate a timeout.

d. Automatically bind termcap key sequences for page-up and page-down to
   history-search-backward and history-search-forward, respectively.

e. There is a new `fetch-history' bindable command that retrieves the history
   entry corresponding to its numeric argument. Negative arguments count back
   from the end of the history.

f. `vi-undo' is now a bindable command.

g. There is a new option: `enable-active-region'. This separates control of
   the active region and bracketed-paste. It has the same default value as
   bracketed-paste, and enabling bracketed paste enables the active region.
   Users can now turn off the active region while leaving bracketed paste
   enabled.

h. rl_completer_word_break_characters is now `const char *' like
   rl_basic_word_break_characters.

i. Readline looks in $LS_COLORS for a custom filename extension
   (*.readline-colored-completion-prefix) and uses that as the default color
   for the common prefix displayed when `colored-completion-prefix' is set.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/