Re: [Sks-devel] 3 million keys & and community help requested

[John Clizbe]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256

Robert J. Hansen wrote:
> On 11/2/11 5:49 AM, Andrey Korobkov wrote:
>> Is there any chances that SKS would be ported to some other, more
>> common databases like MySQL?
> 
> Berkeley DB isn't quite ubiquitous, but it's close.
> 
>> What is the reason behind choosing BDB as a storage?
> 
> It is exactly as much hammer as we need for the nail.  MySQL and its
> ilk are tremendously capable databases that can do everything up to and
> including preparing your morning latte.  We don't need those
> capabilities.  Berkeley DB is much more limited, basically working on
> key/value pairs... which happens to map quite neatly to our problem domain.

I'd have to go look to verify, but I believe BDB was the storage mechanism of
the more widely deployed (at the time of SKS's introduction) keyserver, PKS. One
would need to ask Yaron to verify why BDB.

I do not believe there are any public PKS servers around any longer -- they've
either converted to SKS or gone away.

According to the nice folks at Oracle, there's a compatibility layer now between
BDB and SQLite. May want to start there. :-)

>> I think, having SKS work with client-server databases rather than
>> file-based could give it more scalability...
> 
> I hate to sound harsh, but in my experience the overwhelming majority of
> the time complex systems do not act in ways that conform to our thoughts
> and expectations.

Which is why I forwarded Yaron's initial email(s) describing why SKS, and how
SKS works.

Indeed, IME the single worst bottleneck in SKS is the initial loading of the
database from a keydump. One of my development boxes actively syncs with several
SKS peers. It runs quite well on a 500MHz Sun Blade 100 with ATA/33 disks and
2GB of RAM. [Takes a while to load though :-(  ]

Next up is WAN bandwidth, but that's usually only an issue when a server
initially peers or comes back online after a long absence, and only the first
recon peer will see the slow down.

> If you believe this is a good idea and worth pursuing, I'm sure many
> people on this list would welcome performance metrics between Berkeley
> and MySQL/MariaDB/Postgres.

Yep.

- -John
- -- 
John P. Clizbe                      Inet: John (a) GingerBear DAWT net
FSF Assoc #995 / FSFE Fellow #1797  hkp://keyserver.gingerbear.net  or
     mailto:address@hidden

Raise your hand if you know someone who is alive only because you
did not want to spend time in jail

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12-svn5502-2010-12-23 (Windows XP)
Comment: When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl!
Comment: Be part of the £33† ECHELON -- Use Strong Encryption.
Comment: It's YOUR right - for the time being.
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOsZVGAAoJECMTMVxDW9A04UUH/2mL7YtCYEL6wqvzJ7PARIjG
04Nm9HJpngwyEKANYhvJ5tA93nY7RqeICiqa7oPJwY8VtVT6borIyQV2vndDqNBk
PPBYXhKla+7M52/7nnk6aJ+CtdttpjqP0IqC6tEwO3CLRFyHaASXWVbOscogS5XE
rPRRkjAjD8kYsUnvYH4cghK4dKLV9BsFMj7hwocEwRf6Dkhi8XAWY3EfS0hKmdhI
or+k2R1p3oJM7wbUApYdiYf8sXsbXPoERZx7R3UAoaxR6Wyrrv3oQpwTfgCx0DtG
ACDdrRrj5+Slddu6E5MsvLDSCUh3wBd3Fktj+xT0R3OPIyOD+B4rvSLJ43ZF/XuI
XgQBEQgABgUCTrGVRgAKCRDrXhnz1laYJS2WAP9dZ8w6MMXo06IRbcyLs4up9WJw
lHv+TyArjaHdx062ZwD+J8AIinXm+0W4b5zvhkvMsxcew6YEsiofoChTR4Vvwvs=
=s06f
-----END PGP SIGNATURE-----