[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Monotone-devel] Re: [PATCH] pre-cache database file to improve resp
From: |
Nathaniel Smith |
Subject: |
Re: [Monotone-devel] Re: [PATCH] pre-cache database file to improve response time |
Date: |
Mon, 20 Mar 2006 23:12:18 -0800 |
User-agent: |
Mutt/1.5.11 |
On Mon, Mar 20, 2006 at 07:10:54PM -0800, Joe Wilson wrote:
> >> BenoÃt Dejean wrote:
> >i think this is irrelevant. sqlite does random writes/reads. moreover,
> >if you cat'ed your dvd iso before burning it, that wouldn't speed at all
> >the burning process. but you get the point, if you're running out of
> >ram, the kernel destroys cache pages that have just been read by your
> >'preload' :)
>
> The DVD image (which is several times the size of most machine's
> RAM) had to be read into memory in the first place in order to be
> burned to the DVD. Pages will be evicted and replaced as result
> (all be it at a slower pace). But your OS still manages to recover
> somehow.
Well, no, assuming a reasonably smart kernel, it will notice that you
are doing a linear read and aggressively drop the old pages, with the
result that mostly your DVD reader will just re-use the same pages
over and over without affecting the rest of the cache.
Some other considerations:
-- some people have quite large databases; significantly larger than
100MB. Even significantly larger than main memory, in entirely
plausible cases.
-- there's really no reason to have any confidence that this will
help more than it hurts; certainly it helps you on your
benchmark, but there are a vast number of situations out there,
and my experience with this sort of trick is that it's pretty
random and unreliable; sometimes it can cause really
disproportionate slowdowns. For this case, how do we know?
(Random example of a different situation: running monotone on a
multi-user system.)
-- 'checkout' is the best case for this patch, and a fairly uncommon
operation; the patch actually pessimizes for 'status', 'diff',
'commit', etc., which are far more common. (Because these are
bound by reading the workspace, not the db, and this patch
will tend to trash the OS's cache of the workspace.)
-- we're putting significant work into making monotone's disk
format less seeky, which is the real solution to these sorts of
issues, so in the best case this is only a short-term patch
anyway... if that turns out not to be the case, then hacks like
this can be re-evaluated.
-- it's "bad style" -- which, regardless of whether it works or not,
tends to give people a bad taste in their mouth, and bias them
against monotone. I think there's a reasonable chance that
people would actually rather use a slower program that doesn't
do things they perceive as offensive, than a faster one that
does...
Have you tried running 'vacuum' and 'analyze' on your db? It would be
interesting to know if you still see similar speedup.
You might also try the 8192 page_size... that was committed to
mainline recently as well. (Old db's won't be automatically migrated
and will continue to work fine, anyone who cares can do a dump/reload
to get the new page size.)
Cheers,
-- Nathaniel
--
"If you can explain how you do something, then you're very very bad at it."
-- John Hopfield