Re: [Monotone-devel] Re: [sqlite] disk locality (and delta storage)

[Daniel Carosone]

On Tue, Feb 07, 2006 at 10:17:54AM -0500, address@hidden wrote:
> Something you should experiment with, by the way, is increasing
> the page size so that more records fit on one page and you get
> larger fanout.  Do you get better performance if you rebuild 
> your database with say a 16K or 32K page size instead of the
> default 1K?

We've been discussing a number of benefits that might come from larger
page sizes. It's clear that at 8k, at least, most databases get rather
smaller on disk, presumably because we're packing objects that are
often over 1/2 page in size.  For a number of databases, going larger
than that seems to start increasing the db size again.  

However, you comments about fan-out are very interesting and add
another possible benefit to the list; i'm not sure the performance
implications have been well measured, even at a little extra size
expense. Larger IO sizes should have a notable improvement as well,
assuming most of the data in a page will eventually be wanted (ie,
with good ordering).

[and, to continue the tangent]

> What I'm looking for is a VCS + bug-tracker + wiki that I can just
> drop into a cgi-bin of some low cost web hosting site (like Hurricane
> Electric, which I notice we both use) and have a ready-to-roll project
> management system with no setup or other details to worry about.  Kind
> of a sourceforge-in-a-box, only a lot better than sourceforge.  I'm
> looking for something like monotone with CVSTrac enhancements that
> works over HTTP.  Nothing like that currently exists, I'm afraid, 
> so I've been working on my own.

There are several of these sorts of things around, including some
which might fit well enough with monotone, with varying amounts of
work.

However, I've yet to see anything like what *I'd* want out of such a
system.  It seems a little odd to me to build a centralised, online
information system for tracking state and documenting activity around
and about source code in a distributed and disconnected VCS.  

It seems to somehow miss the point, or miss the opportunity to be
truly useful to developers for whom the monotone operating model is
naturally attractive already.

What I'd really like to see is something that, instead of just
plugging into monotone to show source code state and patches, actually
used monotone for all storage and 'information transport', and allowed
developers to update the wiki pages and bug tracking information in
the same way they can update the code: offline, with syncs and merges
later as needed.  

Wiki pages doesn't seem so hard, they're pretty much text documents
stored in a VCS anyway.  Bug tracking state would require a little
more smarts, with suitable text formats stored in the VCS and
code/hooks to assist in making sane merges between multiple heads of a
given ticket, but even that doesn't seem terribly hard.

It could still be a web ui if people find that comfortable, just one
that developers would often run pointing the browser at a local
server, with a commit at the end of a session, and a later sync and
merge. Of course, you could/would always have an instance of this
running on a public webserver in a well-known place, just like
monotone projects typically do for their source databases: for
convenience, rather than necessity.

Some of these things might eventually go into monotone itself, perhaps
building more tools for project policy and practice assistance around
base mechanisms such as certs and the DAG structure.  More of it could
be done as additional programs, with libraries of hooks to install as
appropriate to help get their stuff done. We already have the
beginnings of several of these things, such as testresult certs, but
they're primitives waiting for further development as projects
discover practical and creative ways to use them.

There might be value in this integration: perhaps having a single
commit that fixes a bug and also updates the tracking entry for that
bug can save some of the more manual correlation and entry work that's
otherwise necessary for good data quality, and even assist tracking
when the fix is later propagated to other branches.  Perhaps storing
some of this state directly on the revisions, as certs in a structured
format the tracking tools can interpret, will have similar benefits.

> Monotone is my favorite of the current crop of VCSes by the way.
> It has 80% of what I am looking for.  What I'm really after is 
> a better monotone.  I have been greatly inspired by your work,
> as have others, I notice.

Yes, we're all interested in a better monotone, both for more
immediate and direct VCS purposes, and for some of the more
speculative and far-reaching implications that hopefully will build on
that later, as you can see above.  I certainly share your sentiments.

--
Dan.

pgpWzST3OLE4i.pgp
Description: PGP signature