Re: [Gnu-arch-users] Re: in-tree pristines fatally wounded (merge-fest etc)

[Tom Lord]

    > From: "Samuel A. Falvo II" <address@hidden>

    > On Wednesday 03 December 2003 08:55 am, Tom Lord wrote:
    > > It seems to me that the larger community and the vendors are in a kind
    > > of symbiotic relationship these days.  And everyone knows that,

    > Unless you're a customer of Microsoft.  

Well, we are talking about the free software world.


    > This leads me to question what the business use of arch could be.  Right 
    > now, the CVS and SVN management models are perfect fits for businesses 
    > looking to use some kind of source code management tool.  Traditionally, 
    > corporations prefer hierarchy.  Their is the CEO, and under him, the 
    > CTO, COO, et. al.  Under those, you have your senior level managers, and 
    > under them middle managers, then division managers, then ... ad nauseum, 
    > until you reach `the shop floor.'

Sorta.  There tends to be a richer structure than just a simple
hierarchy.  There's some legal stuff that is strictly top-down, then
there's various ways to organize management tasks, then there's all
the unofficial alliances and animosities and so on.  But ok, as a
first approximation (and very literally in some places) that sounds
about right.


    > It follows that businesses expect this same kind of support from their 
    > source code management software.  What appeals to many businesses is 
    > that CVS and SVN have a *single* repository, which *one* person is 
    > *responsible* for maintaining.  If something ever happens to that 
    > repository, that one poor soul is in for it by management.  E.g., 
    > hierarchy leads naturally to paper trails, even if the paper is virtual, 
    > and paper leads naturally to chains of command and responsibility.

I think you're being a little too literal there asserting that
centralization of responsibility for source code assets implies, for
example, the need for a tool that literally uses a single server for
that asset.  But yes, I would agree that companies _tend_ to want such
things as:

        ~ a central point of authority for making and executing 
          source code managment policy

        ~ a "captain's chair" view of the state of all the source 
          code;  the ability to examine ongoing workflows, to make
          global queries of the source base crossing organizational
          boundaries, and yadda yadda yadda

It's often not perfect.  Sometimes groups go off and "do their own
thing" source code wise.  Sometimes there's astronomical amounts of
source code sometimes coming from business partners -- little opportunity
to centralize source management technology and a volume of data so
large that no meaningful "captain's chair" view is possible.

But sure, again at least as an approximation, the centralized elements 
are important.

(It's interesting, in that light, that Collabnet isn't (as far as I'm
aware) selling "Subversion Services" -- they're selling a
broader-scoped (purported) total solution for software and engineering
process management of which Subversion is aimed at being just a
component.)

But why would you think for even one second that arch doesn't support
that tendency to want central management quite well?  Better, in fact,
than CVS and SVN in terms of scalability, ease of ensuring data
integrity, flexability and precision of tracking work-flow,
expressiveness and power for implementing source management policy,
....

Seriously, I don't see what you think the problem is.   That there's
more than one physical archive?  So what?   That's a storage
management and namespace partitioning issue and it's not even all 
that different from CVS and SVN:  I doubt that in a shop as large as
what you describe that people would use literally exactly one svn
database or CVS repository.

And there's another strength of arch: that you can fluidly manage
those administrative boundaries between archives yet _also_ preserve a
fully integrated revision control environment that binds them
together.

To be sure: arch is _also_ great for a rigorously _decentralized_ and
anarchic process.  That too, is one of the things the design is aimed
at not least because when it manages to occur in the free software
world, often that's a good thing (and, without that support, the free
software developer community is vulnerable to certain kinds of
parasitic attack <cough>BK</cough>).

But that arch can answer both of these needs (and even bridge between
them) just means, I think, that arch has really struck a chord and
nailed some fundamental functionalities.
        

    > arch encourages a development model which is antithetical to the
    > way most businesses work, I think.  Support for star-merging
    > greatly reduces the political influence of the "repository
    > maintainer," as people in the development group can star-merge
    > directly from each other, completely circumventing the
    > repository maintainer.  Then, when the repository maintainer
    > needs to perform an integration, there may be a module that
    > (s)he doesn't want star-merged with something that (s)he does
    > want, which can lead to increased integration times, and woe is
    > he, additional work to weed out the changes (s)he doesn't want.

I agree that arch encourages a development model antithetical to the
way _many_ businesses work: specifically, arch encourages _having_a_
_development_model_.

When you talk about the "political influence of the `repository
maintainer'" ..well, hmm.   One _does_ hear reports of Admins Out Of
Control who set up the tools environment as their own kind of castle
-- protecting their job more than helping development proceed.   And
yes, there can be a political battle there to move the shop of such an
admin to a better state -- especially if that means deploying tools
that the Admin Out Of Control doesn't know very well and tools that
simply don't need the Admin quite so much to keep working.

But there's also a less cynical view of the "repository maintainer" --
the gatekeeper view.   A gatekeeper is kind of a Maxwell's Daemon who 
controls the doorway to the integration/production/release versions of
a project's source.   Linus is an example of a gatekeeper and the
linux kernel project illustrates another important principle:  that
often it is very productive to have a _tree_ of gatekeepers (Linus is
at the "root" of his tree) -- letting software "filter up" to the root
of the tree.   (And isn't it strange, apropos nothing, how in computer
programming the "roots" are usually "up" and the "leaves" usually
"down"?  But anyway...)

The same merging features in arch that you correctly point out can be
used to implement hopeless chaotic messes -- those same features can
also be used to implement gatekeeper practices with unprecedented ease
and precision.  That's what they (especially star-merge) are there
for.

A hammer can both build and destroy a house -- in spite of that
duality, it's still a good tool to have around.


    > You know and I know that this is a bogus excuse.  But managers don't know 
    > this, and *won't* know this.  They'll quite often have nothing to do 
    > with the idea of thinking about things differently[1].

I'm going to move your footnote up here:

    > [1] In fact, many managers positively refuse to use anything other than 
    > CVS, 

and many refuse to use anything other than Perforce, or ClearCase, or
whatever.  And sometimes, indeed, for quite irrational reasons --
sometimes, on the other hand, because so much about those systems is
already a commodity, perhaps one the manager already has experience
managing.


    > since it has per-file operation, rather than per-tree.  

Perhaps some.   I doubt that that is _often_ the dominant
consideration (even when its a handy excuse) and, anyway, it's not
quite as meaningful a distinction as you might think at first.


    > This means that even SVN is at a huge disadvantage compared to
    > CVS.  I find this bizarre, since CVS lacks the facility to
    > revert a file to a version prior in its history, while SVN has
    > it easily available.  I had a conversation recently with a
    > friend of mine, and after trying tla, he said, "It's very cool,
    > but it lacks SVN's ability to easily revert a [single] file to
    > an older revision."  That was his one, deciding factor in favor
    > of using SVN for his projects instead of tla.  Personally, I
    > admit that this feature would be nice to have, but is not a high
    > priority item for my immediate needs.

Your friend was simply mistaken.


    > Note that I'm playing with literate programming with qm, partly for the 
    > learning experience, 

Fun and interesting, isn't it?

    > and because I personally feel that many open source projects
    > lack important 'internal documentation' or hacker's guides.
    > This tends to lead groups of developers to form cliques.  I wish
    > to avoid this situation at all costs, as I feel it ultimately
    > hinders open source's progress and acceptance.

You seem to think very "socially" -- you think about the emergent
effects in a society of the actions of individuals.  For a specialized
subset of such effects, that's what Stephen studies professionally.
Some interesting trends in recent philosophy are concerned with that
kind of thing.   I like to try to think that way too.

I agree with what you say there but with a qualification.  The
"cliques" are a dual-edged sword:  they can do some good, too.
You can get kind of a "super-mind" made up of many individual people
where the super-mind understands the program far better than any one
person could, no matter how good the internals documentation.

But there's a flip-side to that flip-side, too, in my opinion.
There's something to be said for "human scale" engineering: when your
program is so complicated that it _requires_ a super-mind to work on
it, then you're in trouble because your program is now well and truly
out of control.  The problem with a super-mind is that its intensions
and proclivities, if it can be said to have those, aren't the
intensions and proclivities of any individual and aren't even a simple
sum of the intensions and proclivities of the individuals.   A
super-mind can "decide" to lead its participants "down the garden
path" and, because its so much smarter than any one person, it's a bit
hard for anyone to recognize when that's going on.


    > Note that I do work (I *really* need to find alternative sources
    > of income; In-N-Out Burgers just isn't cutting it for me) and go
    > to school (Physics degree), so progress on qm is rather slow at
    > the moment.

Yup.  It seems like a pretty difficult period in the computing systems
industry because there isn't a lot of demand for innovation:  people
are still scrambling to catch up with the last round; they're skiddish
about all the now-apparent failure modes; there's been a "successful" 
joining up of technology-to-semi-automate-the-latest-innovations with
market-glut-of-people-qualified-to-drive-that-semi-automated-interface.
So unemployment skyrockets, salaries freeze and fall, and there's
precious little that's _interesting_ to work on.

And as a side effect of that boringly cyclical phenomenon -- we have
something new in the history of computing: abandonment of the
fundamentals.  From the 60s and through the late 80s there were,
within the industry, at least a moderately healthy number of parallel
efforts to innovate by "starting over":  new operating systems, new
interactive environments, etc.   All the Next Big Things over the
years can trace their roots back to such efforts -- even when the
efforts themselves failed (the expected case) they had surprising
"spin offs"

And now that's pretty much gone.  Where there is innovation at all,
it's always in the form of building ever higher towers on top of what
was built-out in the 90s.  So we take, for example, a really baroque
foundation of web services technology and add on top of (or
underneath) that a computing-as-utility architecture or
centralized-management-of-authentication service or
hey-isn't-that-emacs-but,-y'know,-not-as-good IDE.  And so you get a
kind of negative spiral where people are working to make it cheaper to
do what we already know how to do (Today's Innovation) and that
depresses the labor market further (You Want Fries With That?) and
that makes it less necessary to _further_ make it cheaper to do what
we already know how do (Today's Innovation Has Falling Demand)
because, heck, with all that cheap labor who needs additional
technology to make labor more productive?

It's a classic negative-feedback loop.

So, I dunno, in contrast to that my opinion is that some projects to,
oh, say -- build new text editors, or replace libc, or replace all
the standard shell tools, or explore a totally different approach to
GUI applications: those kinds of things have much more plausible
prospects for creating the Next Big Thing then what we see going on in
industry.


"Systems Software Research is Irrelevent" -- Rob Pike, 2000
-t