Re: xmalloc crash

[Eduardo A . Bustamante López]

On Sat, Jan 06, 2018 at 01:42:25AM +0200, Alfred Baroti wrote:
> Hi,
> I found this from long long time ago.
> Is this a serious bug?
[...]

This is not a serious bug at all. It's just a memory allocation failure.

> [root@n1x ~]#su nix
> nix@n1x:/root$ printf "%s\n"
> {{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}
> bash: xmalloc: .././braces.c:793: cannot allocate 7329062664 bytes
> (614682624 bytes allocated)

You are performing a brace expansion which generates 62^5 combinations and
requires bash to allocate 7329062664 bytes (6.83 GiB) from the operating
system. The allocation fails at 614682624 bytes (586 MiB).

Once bash is unable to allocate memory, it will crash (xmalloc is a wrapper
around malloc which terminates execution on allocation errors).

Looking at a program execution trace for bash 4.4.12(1)-release (as distributed
by Ubuntu):

    dualbus@ubuntu:~$ LC_ALL=C strace -fo /dev/stdout bash -c ': 
{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}'
 | cat -n | tail -n6
    bash: xmalloc: .././braces.c:793: cannot allocate 7329062664 bytes 
(613113856 bytes allocated)
    116508      28672 brk(0x254ad000)                   = 0x254ad000
    116509      28672 brk(0x254ae000)                   = 0x254ae000
    116510      28672 brk(0x254af000)                   = 0x254af000
    116511      28672 write(2, "bash: xmalloc: .././braces.c:793"..., 95) = 95
    116512      28672 exit_group(2)                     = ?
    116513      28672 +++ exited with 2 +++

You can see that the brk() system call actually succeeds.

So the allocation fails, even if the system has enough memory to handle it.
Upon closer inspection:

    dualbus@ubuntu:~/src/gnu/bash/lib/malloc$ cat -n malloc.c | sed -n 
'777,781p'
       777        /* Silently reject too-large requests. */
       778        if (nunits >= NBUCKETS) {
       779            abort();
       780          return ((PTR_T) NULL);
       781        }

The internal_malloc function that bash uses on some systems (instead of the
libc provided malloc), including Linux, has a special provision that rejects
memory requests that are too large (NBUCKETS = 30).

So you have two options:

(1) Compile bash by providing `--without-bash-malloc' to the `configure' script
(2) Find another option to generate all these permutations that doesn't require
requesting >580 MiB at once.


Question for Chet: Is there a reason we use bash-malloc in Linux systems by
default, instead of the typical glibc malloc which is available on most of
them?