Bash performance when declaring variables (was: Re: [Help-bash] make function local)

[Eduardo A . Bustamante López]

On Thu, Apr 16, 2015 at 11:07:34AM -0400, Chet Ramey wrote:
[...]
> I knew that rang a bell somewhere.  mt_hash is a function in the bash
> malloc library that keeps track of all allocations and deallocations in
> a table.  It's part of the debugging that is enabled when you build from
> the devel code.  It's been well-known for a long time that the debugging
> code in malloc slows bash down considerably; that's why it's not enabled
> as part of bash releases.

Actually, this is the post that motivated me to look into this:
(yes, the conclusion is idiotic, but I guess the rest of the post is pretty
okay).
http://spencertipping.com/posts/2013.0814.bash-is-irrecoverably-broken.html

Now, there is some truth to what he says:

dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<1000)); do 
declare a$RANDOM$RANDOM=1; done' 
./bash -c 'i=0; while ((i++<1000)); do declare a$RANDOM$RANDOM=1; done'  0.01s 
user 0.06s system 93% cpu 0.077 total
dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<10000)); do 
declare a$RANDOM$RANDOM=1; done'
./bash -c 'i=0; while ((i++<10000)); do declare a$RANDOM$RANDOM=1; done'  0.16s 
user 0.48s system 98% cpu 0.643 total
dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<100000)); do 
declare a$RANDOM$RANDOM=1; done'
./bash -c 'i=0; while ((i++<100000)); do declare a$RANDOM$RANDOM=1; done'  
15.44s user 6.51s system 99% cpu 21.959 total

I built bash like this:

CFLAGS='-pg -g -O0' ./configure --silent && make -sj4 DEBUG= MALLOC_DEBUG=

To make sure the malloc debugging code doesn't interfere.

I got a gprof profile with that last run, which gave:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 71.42     12.07    12.07  1100104     0.00     0.00  hash_search
 21.18     15.65     3.58   275435     0.00     0.00  morecore
  1.63     15.93     0.28  6800525     0.00     0.00  internal_malloc
  0.71     16.05     0.12  6200116     0.00     0.00  internal_free
  0.59     16.15     0.10   300001     0.00     0.00  expand_word_internal
  0.24     16.19     0.04  6800474     0.00     0.00  sh_xmalloc
  0.18     16.22     0.03  7203779     0.00     0.00  is_basic
  0.18     16.25     0.03  1932530     0.00     0.00  is_basic
  0.18     16.28     0.03   200002     0.00     0.00  subexpr
  0.18     16.31     0.03   100008     0.00     0.00  find_special_var
  0.15     16.33     0.03                             pagealign

Notice how it spends most of the time in these two functions. Yeah, it's not
mt_* like I said, because I did this a time ago and forgot to take notes.


Does this matter much? I don't know. Having 100,000 variables declared does
seem like something stupid. Still, it shouldn't have that quadratic increase in
performance (I didn't even try for 1,000,000 because it was very slow), because
it is a hash table.

-- 
Eduardo Bustamante
https://dualbus.me/