[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Bash 5 increase in RAM from loop with many interations that append t

From: Scott Kostyshak
Subject: Re: Bash 5 increase in RAM from loop with many interations that append to file
Date: Tue, 27 Oct 2020 12:55:16 -0400
User-agent: NeoMutt/20200925-17-dd27f5

On Tue, Oct 27, 2020 at 09:40:53AM -0400, Chet Ramey wrote:
> On 10/27/20 12:11 AM, Scott Kostyshak wrote:
> > The following example uses more peak RAM on new bash versions than old 
> > versions:
> > 
> > for i in {1..1000000}; do
> >   echo "${i}" >> example.txt
> > done
> > 
> > By measuring peak memory usage with time (/usr/bin/time -f "%E %P %M"),
> > I get that newer versions of Bash use about 284M, where older versions
> > use about 191M.
> > 
> > Is this perceived increase in memory usage worth looking more into or is
> > it intended?
> My guess is that the huge list that results from the brace expansion caused
> the bash malloc to cross over into a larger memory bin because of unrelated
> memory allocation patterns that changed between bash versions. Once you
> allocate that much memory, a binary bin allocation method is going to waste
> some. In bash-5.0, that large allocation is going to use mmap, which adds
> some overhead of its own.
> If you can use valgrind or a similar tool to identify a memory leak, that
> would be useful.

Thanks for the explanation, Chet. I would normally jump at the chance to help 
out because I enjoy digging into things like this, but these days I just can't 
find the time. I will come back to this at some point, but possibly not for a 
while. I'm sorry about that.

Thanks again for all of your work on Bash!


reply via email to

[Prev in Thread] Current Thread [Next in Thread]