[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: strcache scaling issue
From: |
Paul Smith |
Subject: |
Re: strcache scaling issue |
Date: |
Mon, 21 Feb 2011 02:30:28 -0500 |
On Sun, 2011-01-23 at 09:59 +0100, Ralf Wildenhues wrote:
> * Paul Smith wrote on Sat, Jan 22, 2011 at 10:27:01PM CET:
> > Thanks for your investigations here; that's very helpful. I had also
> > had some thoughts about improving the strcache in various ways. For
> > example, I was thinking maybe about splitting it into two: one for
> > filenames and one for variable names, since these strings very rarely
> > overlap.
>
> But a hash doesn't degrade in quality just because you put two different
> kinds of things in it. I wouldn't go for more complexity here. On the
> contrary, if you want an example where one big hash table is used for
> several different kinds of objects very successfully, look at git.
Hi Ralf. I promoted a rework of strcache.c. For your simple scaling
test on my system with debug compiled I get these results:
Build Info
1000
2000
4000
3.82 -g
2.61s
8.85s
33.52s
3.82 -O2
1.90s
7.62s
27.82s
New -g
(with
asserts)
1.03s
2.31s
5.79s
New -O2
(no
asserts)
0.65s
1.50s
3.52s
Not exactly linear yet but much improved.
However there are other issues I've seen. First, I've discovered far
too much is being added to the strcache; all the temporary strings found
during implicit rule evaluation, even those which do not match, are
being permanently stored, which is bad!!
Second, related to my comment above on splitting into two caches: I
think I have a reason to want to do this. On some types of make
compiles, we need to match filenames case-insensitively but we never
want to match variable names insensitively...!
So more work to come in these areas (among others).
--
-------------------------------------------------------------------------------
Paul D. Smith <address@hidden> Find some GNU make tips at:
http://www.gnu.org http://make.mad-scientist.net
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
- Re: strcache scaling issue,
Paul Smith <=