Re: [Sks-devel] Adding DB_INIT_LOCK to sks-keyserver (revisited)

[Jason Harris]

On Fri, Feb 26, 2010 at 09:58:17PM +0000, Kim Minh Kaplan wrote:
> Jeff Johnson writes:

> > From the 3 deadlocks I've seen, I'd guess that any moderately large
> > (>500 keys) is likely to encounter a partition tree deadlock if
> > DB_INIT_LOCK is enabled.

Fortunately, I don't have these problems with pks even though mine
still uses BDB 2.x.  I know Hironobu (OpenPKSD) and Jonathan (onak)
report massive speed slowdowns with SQL-everything (that they used),
but I don't recall if they tried sqlite3.  I believe that BDB is a
solid DB with proper use of the API, having seen it in pks, but am
very hopeful that SKS will do quite well when converted to sqlite3.

> > Meanwhile, db_recover is only as good as the logs that are present,
> > often only back to the last hourly checkpoint. If logs are being

> This is not my understanding of the phrase "All committed transactions
> are guaranteed to appear after db_recover has run, and all uncommitted
> transactions will be completely undone"[1].  As I understand it, after a
> db_recover the database should contain *everything* that has been
> committed.  Regarding automatic removal of log files, I hope you are
> refering to "log files that are no longer in use"[2] in which case they
> are not needed for normal db_recover operation (i.e. to recover from a
> crash of SKS).  They are only needed when catastrophic failure occur but
> I do not think this what we are talking about.

Unfortunately, it remains way too easy to induce catastrophic failure
(e.g. "page not found" - mentioned earlier in this thread) given SKS'
original/existing/unchanged way of calling the BDB API.

In my experience, extraneous logs (those which could have been removed
but were not) only serve to slow down the recover process.  Often, run-
ning a db_archive after a crash will greatly speed a db_recover.  OTOH,
deleting all logs and/or the shared memory backing files to fix a "last
LSN" error is often pointless and ends in an all-too-familiar "page not
found" failure.

-- 
Jason Harris           |  NIC:  JH329, PGP:  This _is_ PGP-signed, isn't it?
address@hidden _|_ web:  http://keyserver.kjsl.com/~jharris/
          Got photons?   (TM), (C) 2004

pgpN8xKOfXyDe.pgp
Description: PGP signature