Re: [Gnu-arch-users] arch and linux 2.7

[Tom Lord]

        Pau:

        > Does anybody think that we should try to do something RSN to
        > promote the adoption of arch in the development of the linux
        > kernel? Linux 2.7 is about to appear and it would be great
        > if new developers could adopt arch as their default SCM.

        > [....]

        > The steps to take, a grosso modo, would be:

        > -stabilize the tla commands
        [...]
        > -add performance hooks addressed to expert users (sorry for the 

        > self-promotion, it's not on purpose :)
        [...]

        > -master archive (or repository):
        [...]

        > -write a HOWTO-arch-linux-kernel:

        > -try to approach several developers (maybe the current
        > kernel developers using arch would be the best fitted for
        > this) to expand its use. 

And others have their own things to add to the list.

Parts of these lists are certainly "things which will be improvements
to tla":  perfectly fine things to work on.

However, I think we need to be cautious on a few points:


1) We shouldn't presume to know more than we know.

  For example, it's one thing to say "we ought to improve some of the
  error messages" but another thing entirely to say "in order to 
  promote kernel-project use of arch, we ought to improve some of the
  error messages."


2) We should look for a real task, not a general idea.  We shouldn't
   be "promoting for kernel use" -- we should be asking "who is our
   Linus?"

  Let's take a lesson from Larry McVoy here.   According
  to his account of history, sure, he wanted to make BK 
  suitable for kernel work in general -- but he very cleverly /
  luckily arranged to make a much more specific goal: to make
  BK work well for Linus.

  I think it's a safe bet that in the early stages BK had (and
  may still have) an ample supply of "rough edges" of its own.
  Yet the perception a short time later is that it is a very
  polished system, rich in features.

  I don't think that transition happens when a project says "Ok, let's
  fix everything we can think to fix and add every feature we can
  think to add."   Instead, I think it happens when the project gets
  to a stage where it's "mostly ready" for use in some situation, and
  then finds some cooperative users committed to using it in that
  situation.   The demands of those early adopters tell you where to
  "selectively polish" and, by omission, tell you what the priorities
  _aren't_.

  The end result is that you have a system that's slick and featureful
  _in_the_areas_that_matter_most_to_users_:  When the next generation
  of adopters show up, they tend to use it the same way as the early
  adopters and so they experience all of the polish and none of the 
  neglected rough edges.

  Who, then, will be our Linuses?  Who are the kernel people willing to
  be early arch adopters, to put with stuff, and to work with the arch
  developer community to cover that last mile?

  What requirements do we want to have for "arch's Linuses"?   Do we
  want people who write a lot of original code?   Who process a lot of
  patches from other people?  Who cherry-pick from various popular
  trees?   Who maintain a down-stream tree?   What?

  If we find "arch's Linuses", how do we want to implement support for
  them?  Do we want to set-up a non-public mailing list?  An issue
  tracker?  Among people interested in helping to develop arch, who is
  volunteering to support our Linuses and to what degree?  If an early
  adopter says "I really need a feature like X" what resources do have
  and what processes do we want to see to it that X is provided?

  The nightmare scenario -- the thing I suspect can not possibly work
  (based on past experience) -- is to just start posting to lkml or 
  /. or whatever saying "Hey, soup's done, come and get it."  That
  never works.

  Sadly, I think that it's entirely possible that Larry has sucked
  that well dry.   You know, there's Linus and then, just as good as
  him for this purpose there's..... um.... there's..... uh....  Hmm.

  "Promoting arch for linux kernel work" might not take the form you
  expect.  Who knows: it might mean inviting Linus over to sprawl out
  on your living room floor, sipping wine and munching cheese, going
  over the "design" of arch and attempting the impossible: asking to
  make a principled commitment to invest in the effort it will take to
  switch from BK to a free software solution.


3) "Who are your friends?  Who are your enemies?" -- Chairman Mao

  One "vibe" I consistently get from the list is that perhaps we don't
  want a Linus at all -- perhaps we want a "grass roots" movement:

  One idea is to _not_ focus on "converting projects to arch".   When
  projects go that way, great!   Let's help that work.   But it's a 
  ridiculously uphill battle to make project-conversions the main aim.

  An alternative is to focus on _non-maintainer_contributors_, in
  which regard projects like cscvs are right on the money.

  In terms of the "list of things to fix in arch to promote kernel
  usage", one of the items was roughly "make a master repository with
  a more complete history".    Hmm.   Well, perhaps -- but it's not at
  all clear to me what that would be a priority need of many
  contributors.    They don't need complete history -- they need a way
  to make a personal infrastructure in which they can cheaply and
  quickly turn themselves into leaf-nodes in kernel project patch
  flow.

  A master repository with slavishly converted history-of-the-world
  _might_ help make those "leaf node infrastructures" easier to set
  up, but it's not obviously the faster, cheaper, better approach.
  The master repository might just be make-work.

  Perhaps, for these non-maintainer contribuors, the high-level goal
  to focus on might be expressed as a world in which you could run 
  a command like:

        % i-wanna-work-on  linux-kernel
        [churn churn churn]

        % gimme-a-tree-for linux-kernel
        [churn churn churn]

        % ok-my-tree-has-a-hack-now
        [churn churn churn]

  the last step invoking tools that guide me through making sure my
  hacked tree is up to date, make it trivial to submit the patch
  (perhaps even submit for me) and so forth.



4) Scale, scale, scale.   But in which direction?

  You know, as far as I can tell, arch is just sleek-as-a-slick for
  projects at the scale of, well, the sum of all my projects.  I have
  outdated hardware, a slow modem connection to the net, rely on some
  pretty minimal archive hosts, don't have someone to administer my
  archives for me -- in spite of all that, it Just Works and
  comfortably so.

  Let's see, that's a paltry 3K files (not counting {arch} or anything
  under it), 32MB tar bundle.  A number of "most active" contributors
  you could count on your fingers.   Pretty simple patch flows.  Easy
  release management process....

  GCC is about 5x as many files (just looking at what's in a relese --
zz  there are separate components that add to this), O(10x) as many
  "most active contributors, only slightly more complicated patch
  flows, ~5x larger tar bundle, faster commit rate though I have
  misplaced my notes that estimate it....

  In another direction:

  Savannah hosts (on rather unremarkable hardware) almost 2000
  projects -- no idea what their cumulative commit rate is or
  rate of burning up admin hours.   SourceForge is presumably a bit
  larger.  

  We can (or can we?) only speculate about various private
  environments: what the vendors use for their GNU/Linux
  distributions, what's going on in various "programming in the large"
  projects with O(1Tb) of source......

  And in another direction:

  What does release management look like at a vendor?   What about
  patch flows?   What are the bottlenecks that impact business
  decisions?   We know externally that they work concurrently on
  multiple product lines and do a lot of testing (hence, inferably, a
  lot of patch propogation).   What's the bookkeeping overhead here
  and how is it implemented?  (My guess: it's pretty simple minded and
  ad hoc, relying on internal project managers to keep in sync with
  one another and manage patch flow "by hand".)  

  And in another direction:

  Has anyone other than me peeked at threads on the Savannah list
  involving GvR and his trials and tribulations regarding revision
  control at the new start-up he's joined?  Fairly typically (as far
  as I know) it seems the start-up started as a few geeks with some
  technology plus a skeletal executive team, got first-round VC, and
  is now fleshing out the corporation.   Part of that is bringing in a
  director or VP of engineering -- who then works on the
  infrastructure for the company.   GvR is a svn fan;   it's a
  multi-platform (msft windows and unix, at least) shop and svn is
  very attentitve to that.   Nevertheless, the last report was that 
  the choice would be Perforce.   One gets the impression that when
  buying from the proprietary revision control vendors one isn't just 
  buying a piece of revision control software: one is buying an entire
  "engineering management process" complete with a commoditized market 
  for things like admistration talent.   Rational is quite clear on
  that point in their marketing literature -- they even have a service
  under which they come into your shop, evaluate, and
  grade-with-recommendations your engineering processes.
  

  In short, there _is_ a next level of challenges out there: in fact,
  there's _many_ next level's of challenges.  There's kernel-sized
  trees, sure.  There's also larger numbers of contributors, higher
  patch rates, higher down-load rates, projects for which browsing is
  more important, projects needing tighter integration with
  bug-tracking and testing, hosts wanting to administer thousands of
  archives for thousands of projects, patch-flows to downstream
  consumers that are far more complicated, "complete packages" from
  the management perspective, and on and on.

  I think we have more or less all the parts of the engine for those
  scalability challenges -- a really good engine, a big beefy V-8
  designed with WWII-era technology yet, miraculously, getting
  incredibly good gas milage.  But the parts are currently spread out
  on a big drop-cloth on the garage floor.   Logically, it's time to
  start putting them together.

  But there's a problem: we don't have a target.  We don't have a test
  track to take the beast out on and make sure we've actually put it
  all together correctly.  We don't have a specific, incented, goal
  for which "next level" to hit.  There's too many choices here about
  what to build (do we put in the fuel injectors or leave them out?
  do we use this carburator or that one?).  Absent a specific target
  for which there's a clear need, and please forgive the shifting
  analogy, the risk here is that we wind up assembling a Spruce Goose.

-t