Bash regexp parsing would benefit from safe recursion limit

[willi1337 bald]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux DESKTOP-0TQV3NS 5.10.60.1-microsoft-standard-WSL2 #1
SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.1
Patch Level: 16
Release Status: release

Description:

A deeply nested and incorrect regex expression can cause exhaustion of
stack resources, which crashes the bash process.

Repeat-By:

$ cat regexsegfault.sh  ; gdb -q --args ./bash regexsegfault.sh

#!/bin/bash
A=$(printf "%32768s");
A=${A// /(}
[[ "B"  =~ $A ]]

Reading symbols from ./bash...
(gdb) r
Starting program: /mnt/c/Users/Admin/bash-5.1.16/bash ../regexsegfault.sh
[Detaching after fork from child process 13645]

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7e808ad in parse_expression (regexp=regexp@entry=0x7fffffffd880,
preg=preg@entry=0x7fffffffd980, token=token@entry=0x7fffffffd870,
syntax=syntax@entry=242428,
    nest=nest@entry=10914, err=err@entry=0x7fffffffd868) at regcomp.c:2249
2249    regcomp.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7e808ad in parse_expression (regexp=regexp@entry=0x7fffffffd880,
preg=preg@entry=0x7fffffffd980, token=token@entry=0x7fffffffd870,
syntax=syntax@entry=242428,
    nest=nest@entry=10914, err=err@entry=0x7fffffffd868) at regcomp.c:2249
#1  0x00007ffff7e83931 in parse_branch (regexp=regexp@entry=0x7fffffffd880,
preg=preg@entry=0x7fffffffd980, token=token@entry=0x7fffffffd870,
syntax=syntax@entry=242428,
    nest=nest@entry=10914, err=err@entry=0x7fffffffd868) at regcomp.c:2207
#2  0x00007ffff7e83b66 in parse_reg_exp (regexp=regexp@entry=0x7fffffffd880,
preg=preg@entry=0x7fffffffd980, token=token@entry=0x7fffffffd870,
syntax=syntax@entry=242428,
    nest=nest@entry=10914, err=err@entry=0x7fffffffd868) at regcomp.c:2159
#3  0x00007ffff7e80949 in parse_sub_exp (err=0x7fffffffd868, nest=10914,
syntax=242428, token=0x7fffffffd870, preg=0x7fffffffd980,
regexp=0x7fffffffd880) at regcomp.c:2496
#4  parse_expression (regexp=regexp@entry=0x7fffffffd880,
preg=preg@entry=0x7fffffffd980,
token=token@entry=0x7fffffffd870, syntax=syntax@entry=242428,
nest=nest@entry=10913,
    err=err@entry=0x7fffffffd868) at regcomp.c:2282

Fix:
Count the stack frames during recursive parsing and emit error before stack
resources are entirely consumed.