[Gnu-arch-users] Re: Linus

[Tom Lord]

    > From: "Stephen J. Turnbull" <address@hidden>

    > >>>>> "Tom" == Tom Lord <address@hidden> writes:

    >     Tom> Are Linus' limits on his ability to process email _really_
    >     Tom> supposed to be a bottleneck on linux kernel development?

    > Yes.  "Conceptual integrity ... dictates that the design must proceed
    > from one mind, or from a very small number of agreeing resonant minds."

Who said that?  Yes, I agree with the quote but it doesn't answer my
question (which was poorly phrased -- I see your confusion).

I was _not_ asking: "Do we really want Linus (and the small set of
people with whom he works most closely) acting as an important filter
on kernel development?" 

I _am_ asking: "Do we really want Miles to have to repeat sending a
patch 3 times?"   

Davide pointed out, correctly I think, that there are some good side
effects of the labor intensive "datagram, point-to-point patch queue"
--- but also that better systems are probably possible.

There's a "what's the 80% solution" question.  If the kernel is the
only project in the world, and its scale is pretty much fixed, then
what we've already got is arguably the 80% solution -- further
infrsatructure arguably over-engineering.

But the kernel isn't the only project and its scale is not fixed: it's
not even a majority of the lines of interesting code; the developer
community is growing; and it sure looks as though increments in
infrastructure (e.g., SourceForge) have a huge impact.

Why the particular increments I'm suggesting?  Well, experience with
arch, largely: they're all generalizations and extensions of useful
practices we're seeing emerge already.



    > The thing is, the number of people who think they can stand in for
    > Linus (and making decisions based on the kind of information you put
    > in that report is precisely "what Linus does", although he doesn't
    > currently get anywhere near that much information) is very small.
    > Even among kernel hackers I doubt there are more than a couple dozen
    > who would pull such reports except for the fun of it.

    > Yes, even if available to everybody, that database is really the tool
    > of Linus and his lieutenants.

I'm not expecting to "let 10,000 kernel releases bloom".  A few 10
sounds about right, though, and from what I can tell, is pretty close
to what we actually have.



    > Second, the number of responses logged in your "artist's conception"
    > web page was unrealistic, unless supported by the kind of automated
    > testing I described, with really muscular organizations (like your
    > sgi-skunkteam) running huge numbers of tests. 

I'm not sure what you think is so unrealistic.  I didn't imply the
existence of more than 10 major development lines.  None matched the
requested "2.6 plus these three patches" exactly.  I didn't make
explicit but did mean to imply that the three requested patches aren't
randomly selected from the set of all available but were rather some
that you'd reasonably expect to show up in trees fairly quickly (note
the comments on them).

I did imply a _moderate_ amount of test runs: a nightly testing
service offered by osdl; more expensive tests osdl runs with less
frequency; a local installation of their own variation on the test
framework at sgi and suse; and the view that Linus' review is a kind
of test (though admittedly one that only ever gets "run" for one
development line).

I also implied that all of the returned trees were, in fact, using
linus-2.6-stable as a baseline.

In short, I described something not _too_ far from the current
situation, but further automated and transparent, with better record
keeping, and with a lower barrier-to-entry for new (friendly) forks.

Come to think of it: from what I have seen in job listings and so
forth, there are in fact many more than 10s of kernel forks.  There's
hundreds of them -- just many you don't hear much about.  And we're
just now beginning to see the first hint of the tidal wave of
development I think we can expect from increased global adoption of
free software.  That's a lot of interesting economic activity to
encourage and tools such as I've sketched out would do so rather well,
I think.


    >  The combinatorics (which is spelled E X P O N E N T I A L, as
    > you well know) will kill you.  Such queries will return "closest
    > match is linus-2.6-stable (missing #4598, #987234, #32454)" most
    > of the time.  Most of the rest will return "closest match is
    > andrew-2.6-merge-queue" (missing #4598).

No kidding.   I wouldn't expect that to change in a _huge_ way
although by having a database of patches and tree-states, new degrees
of freedom are opened up.


    > Another way to think about it is that to get that kind of numbers, you
    > (as the author of the query you described) are very likely the
    > debian-unstable maintainer---probably the best way to get experimental
    > kernel patches installed on several thousand machines in one go---and
    > you know the leader of the sgi-skunkteam personally (not surprising,
    > she used to be your boss and wrote those patches), and (because
    > sgi-skunkteam has an annual $5 million budget for Linux development,
    > some of which bought fish and beer for Linus at the last LinuxWorld)
    > Linus actually answers your email personally, on alternate Thursdays,
    > anyway.

    > What has changed?  Not much, except Linus, Andrew, and Alan have much
    > better access to certain kinds of information.

    > Factor of 2 speedup at best.

They have better access to information.  So do all the vendors.  So do
all the major IT departments.  So do all the most serious
contributors.  It may speedup up, Linus, Alan, and Andrew only by 2x
but it also speeds up those other groups _and_ gives them new and
interesting degrees of freedom.

I think you're taking too narrow a view of what's going on the kernel
world when you focus on just a few big names.

-t