Re: Mercurial policies in the core

[Rik]

>
> ------------------------------------------------------------------------
> Subject:
> Re: [OctDev] Functions in the core
> From:
> Jaroslav Hajek <address@hidden>
> Date:
> Fri, 10 Apr 2009 14:02:20 +0200
> To:
> Thorsten Meyer <address@hidden>
>
> To:
> Thorsten Meyer <address@hidden>
> CC:
> address@hidden, address@hidden
>
>
> On Fri, Apr 10, 2009 at 12:45 PM, Thorsten Meyer <address@hidden> wrote:
>   
>> Jaroslav Hajek wrote:
>>     
>>> Maybe we differ slightly in the view of the development archive. IMO
>>> these are just patches that can easily be reverted. I didn't even yet
>>> add the functions to the build process, so that they won't be
>>> installed if someone uses a snapshot - they're just there for
>>> development testing. So I don't really regard my act as true addition.
>>> The discussion you call for has just started :)
>>>
>>> I don't basically object to a policy that the main archive should be
>>> regarded as a more sacred place. But as I have explained earlier, IMO
>>> this significantly clashes with the policy of having a linear archive,
>>> which makes parallel development for a longer time quite difficult.
>>> So, if that's going to happen, I think we should allow merges. I'll
>>> then happily use my experimental repo for most of my development.
>>> For instance, I've added a couple of new functions (and extended some)
>>> without discussing with anyone, mostly for use in other functions. I
>>> hope this is OK, at least nobody complained. Anyone is certainly free
>>> to raise objections to any of my patches.
>>>
>>>       
>> I think we should really come to a common understanding about how the
>> development archive is meant to be used and how "sacred" it is.
>>
>> About sacredness: the faster the development of octave advances and the more
>> contributors we have, I think, the more difficult it will get the avoid that
>> experimental, uncomplete or inconsistent changes accumulate in the 
>> development
>> archive.
>>     
>
> I see no reason why they should accumulate, unless the corresponding
> developer is reluctant to clean up after himself. They will certainly
> happen (and they do happen).
>
>   
>> Once this happens, it could be quite tedious (and not much fun at all)
>> to clean it up again. So I personally would prefer to have clear and rather
>> strict rules about what is allowed to go into the development archive.
>>     
>
> Personally, I felt the ability to push directly to savannah hg as
> important boost for my contributions, because I no longer needed to
> wait until someone, usually JWE, reviewed and approved my patches.
> When I eventually realized that there was generally very little
> opposition to my patches, and that I was able to respond to
> suggestions quickly, I eventually started pushing most patches
> straight away except for fundamental changes.
> It seems to me quite easier to say "get the latest tip to try the
> stack arrays optimizations" instead of "get revision XXX and apply
> patches YYY.1, YYY.2 and YYY.3 from my previous mails to try the stack
> arrays optimizations". At least the first option is much easier for
> people to actually give it a try.
> Developing in separate experimental repo, such as those generously
> provided by Thomas Weber, would be comparably easy to try for anyone
> (you just clone and build), but it's much harder to manage if merges
> are not allowed (see the previous discussions).
>
>   
>> About merges: I don't think that I fully understand what this issue is about,
>> mostly because I don't have any experience with merging large development 
>> trees.
>> Only, I keep reading that distributed version control systems like mercurial
>> make merging much less painful than with older tools. Anyway, this seems to 
>> be
>> about how the development archive can be kept "clean" while at the same time 
>> not
>> hindering the development of exciting new features and performance 
>> improvements
>> in octave too much. So what's the best tradeoff between those two aims?
>>     
>
> I dunno. I think a linear mainstream repo is OK when the policy is
> relaxed - most development will then happen in this repo. Accepting
> merges allows more convenient development in parallel repos, and
> therefore facilitates imposing a more strict policy for the main repo.
> But combining the linear requirement with a strict policy IMHO hingers
> the development pace. That's going back to the situation when JWE
> needed to review and physically apply all patches.
>
> Right now, I try to honor the linearity, so I develop everything in
> small chunks and push them as soon as they're ready, fixing things up
> later if problems arise. Right now I can address issues quickly, so I
> don't see this as a big problem, but that may change.
>
> All that's just my personal viewpoint, of course.
>   
I'm new to both Mercurial and the way it is implemented for the Octave
project so I will say upfront that my critique might change as I get
more experience with both.

The advantages I've seen from using Mercurial would apply to most
distributed SCM packages: 1) blazingly fast command execution because
commands operate locally rather than over a network to a central server,
2) easy creation of sandboxes and the ability to work in parallel not
only with other developers but also with myself.  I use 'hg clone' to
create several sandboxes with one devoted to work on a hot-fix bug, one
devoted to documentation cleanup, etc.

One of the disadvantages I've seen is with the current Octave policy to
maintain a linear revision history.  This makes browsing the history log
easier, which I do deem important, but it limits the advantages of
parallel repos.  In a typical scenario, I clone the repository on day 1,
make some changes, and then commit them on day 5.  In between, another
developer may have committed some changes which have nothing to do with
my work.  Nevertheless, when I push my changes to the central server
Mercurial believes I have branched the code and I am now trying to
re-merge with the main line development.  Mercurial handles the merge
very efficiently because it was designed for just this situation of
multiple repos.  Almost never is there actually a need for human
interaction because most of the time developers working in parallel are
working on different parts of the code base.  It does, however,
introduce a fork and a join in the revision history.  Mercurial's tools
don't care about this branchification of the history.  Bug finding tools
like bisect work equally well on branched and unbranched histories.  It
does make the history a spaghetti of intertwining branches but this can
be overcome by using Mercurial tools such as keyword searching in the
history file.  To find commits related to the current printing issues
one might try 'hg log -k print.m -k pdf' which would find commits
mentioning pdf and affecting the file print.m.

The current linear history is kept in practice by using Mercurial
Queues.  The sequence is 'hg qnew new_patch; make changes; hg qrefresh;
hg pop; hg pull; hg qpush; fix any merge changes; hg qfinish; hg push' 
I now no longer use the basic Mercurial commands 'hg clone' or 'hg
commit' which I find rather strange.  It seems as if we are fighting
Mercurial rather than having it work for us.  To that end, a tool like
CVS, which keeps a linear history by default and requires explicit
commands to create a branch, would be a better fit. (Note: I am not
suggesting going backwards to CVS, only that it is a more natural fit
and that there may be another SCM out there that works more naturally
with linear revision history).  I also find it annoying that after using
'hg qrefresh' to save my work that 'hg status' doesn't naturally show
the changes I have made from the base repository as it would in the
normal Mercurial development case.  One can work around it with 'hg
status --rev -2' but it only underscores the awkwardness.

The current methodology is fine for small numbers of developers but
there is still a race condition between 'hg qfinish' and 'hg push'.  'hg
qfinish' does a local commit and if another developer checks in changes
to the central repository then 'hg push' will fail.  Ordinarily you
could just do a merge but to preserve linear history you will now need
to do more involved surgery on the local repo such as 'hg export -o
latest_change tip; hg strip tip; hg pull; hg import latest_change; hg
push'.  At this point it feels like the SCM is getting in the way of
development rather than being an aid to faster development.

Just my thoughts,
Rik