Re: [Monotone-devel] Question on layering

[William Uther]

On 21/02/2007, at 7:57 AM, Chad Walstrom wrote:

> > Yep, I'm aware of these. The problem is that they all need a
> > terminus running outside the firewall, and also are probably
> > forbidden under our policies as they are generic tunnels not just
> > protocols which happen to run over HTTP.
>
> *shrug* It might be time to start investigating extending monotone to
> present an HTTP interface as well (i.e. mtn serve --protocol http).

It was to get around a firewall that I was using the ssh:// based  
syncing (until I found out is isn't recommended for general use).

Here are some features I'd like to see.  I know I'm new here, so feel  
free to ignore these comments :), but then again maybe a new user  
perspective is useful.  I'll try to scratch some of these itches  
myself :).  I'll try and scratch them myself :).  Possible  
implementations are discussed at the end.

Features I'd like to see for setting up your own repos without lots  
of admin help:

  i) anonymous pull over http(s).  By http(s), I'm referring to an  
already established server.  i.e. I should be able to stick a  
directory of "monotone files" somewhere already published, and people  
can pull from it (efficiently).  I don't want to have to plug in to  
apache, or start a monotone-specific server.  The "pull from http"  
needs to be in the standard client.  The "export to directory as  
repository" needn't be in the standard client.

  ii) anonymous push over email attachments.  Something like "mtn  
push-email repos > attachment".  Then you email the attachment.  The  
receiver uses "mtn read < attachment".  The push-email command needs  
to be in the standard client.  The mtn doing the read will probably  
be the server from i (doesn't need to be the standard client).

  iii) get ssh:// access working as a 1st class system.

Features for interacting with current hosting services:

  iv) A format for storing a monotone repository in a subversion  
server.  This could just be i) as a first pass, but in general I'd  
like to be able to do a full sync with just the normal mtn client  
(possibly only when configured to link against the svn client libs).   
Bonus points if you can do it so that ordinary svn clients still work.

  v) mtn_cvs :)

Other things that would have put me off monotone if I wasn't a VCS  
junkie:

  vi) You need partial pull.  This should be seamless.  By that I  
mean that the 'partial' repository should have a pointer to another  
'parent' repos, and so on.  When one db doesn't have the data you  
need, it should seamlessly fall back to the parent, and the parent's  
parent, etc.  Think of the repository as a local cache, rather than a  
full repository.  In fact, it would be nice if it were possible to  
_optionally_ drop data from the local cache if there are newer  
revisions and it hasn't been accessed recently.  It should also be  
possible to say "always push data to my parent rather than me".   
Finally, you might even want to specify different parents for  
different branches.

  vii) convenience commands.  mtn clone == mtn pull --partial && mtn  
co (and puts the partial repository in the _MTN directory of the  
working copy).  mtn pcmp == mtn pull && mtn commit && mtn merge &&  
mtn push.  mtn pu == mtn pull && mtn up

Possible approaches:  N.B. I don't think that many of these features  
are far away.

  i) "serve over http".  It seems like this is pretty close to  
happening.  There is the n.v.m.dumb stuff.  This doesn't have a good  
readme at the moment.  I've had a brief glance, but I haven't spent  
the time to figure it out properly.  Not understanding it very well,  
I think there is a problem in that 'pull' isn't integrated into the  
normal client, and isn't in a form where it could easily be  
integrated.  Good proof of concept though.

  ii) "mtn push-email" should figure out what to push just like a  
normal push, but then write the packets to std-out rather than  
netsync.  That doesn't seem too hard (he says waving hands and not  
knowing the code at all :) ).

  iii) I think part of what is needed here is testing.  I'll try to  
write some test cases.  I was thinking of implementing an "sh://"  
protocol, which is just like "ssh://" but local.  That should be  
possible to write tests for.

  iv) A normal subversion server has a fixed directory layout with  
"branches/", "tags/" and "trunk/".  If you link with the svn  
libraries, then you could use them to access the server, add another  
directory there, "mtn/".  That would hold a n.v.m.dumb tree.  The  
tricky part is also looking for changes in trunk/ since the last  
change to mtn/ and moving them across into the mtn/ repository.  It  
wouldn't be too hard in a special program, like mtn_cvs, but you'd  
really want it in the normal client so that changes would be synced  
with every sync :).

  v) well, that's already going really well. :)  Just needs to be  
tested and merged.  I've already got some extra test cases here.   
I'll post them in a bit.

  vi) There is a partial-pull branch, but it looks like it has just  
started.  It also seems from the wiki that the concept is more  
"partial-pull once and lose history" rather than "use a local db as a  
cache for a remote db", but I may have misunderstood.  I prefer the  
"hierarchy of caches" approach.

  vii) All of these can be done pretty quickly (even as local scripts).

I was going to start with iii), v) and vii).

Be well,

Will        :-}