[Gnu-arch-users] on RH, part 2

[Tom Lord]

I seem to have (understandably) given some people that I only like to 
rail indignently against RH -- that I don't recognize the value of
their contributions, the challenges of their business model, the
realities of capitalism, or the quality of their hackers.   There is a
sense that I am trying to apply an ethical analysis in a context where
it is inappropriate to do so.

I'd like to put that perception to rest and I've tried to do so by
collecting my thoughts in two parts.   Part 1 is a review of what I
think are the most serious ethical issues with RH's business model --
presented from what I hope is a balanced perspective.   Part 2 are
some thoughts _towards_ a better business model for RH et al.

-t


                   Towards a Better Business Model
                   -------------------------------



* Drop the Barriers

  Does RH actually _need_, at this point in history, to horde software
  and bind users with a proprietary RHN infrastructure?  Do they
  actually _need_ to divorce their distributions from community
  efforts, differentiate by ISV certification, compete for mind-share
  against truly free distributions with Fedora?

  And, on the other hand -- can they afford to _not_ drop these
  aspects of their service?

  It's clear on the one hand that these apsects of their service
  create no value whatsoever for customers.  RHN is valuable to
  customers -- but the legal restrictions that accompany it are not.
  RH distributions are valuable for customers -- but the separation of
  them from the larger community is not.

  It's clear on the other hand that these aspects of their service are
  increasingly -- even pressingly -- vulnerable to competitive attack.
  The technical advantages of RH's distributions compared to more
  free, more communitarian efforts like Debian are, at best, slight
  and diminishing (even though an individual user frustrated by a
  missing driver or slow package update may feel differently).
  Superior infrastructure alternatives to RHN are already on the
  horizon.

  Someone in the free software industry, back in the early days, once
  went around handing out little tickets -- like those you might get
  at an amusement park, each saying

                              Admit One
                                 -*-
                            To the Future

  It was a cute gimmick that gave a handy excuse for talking about the
  inevitability of free software.  Free software is coming whether you
  like or not, the pitch went -- and it's going to win because it's a
  better approach.   Larry McVoy said much the same thing in his now
  famous essay to Sun.

  The pitch was directed primarilly at business decision makers and
  key customers who were used to the non-free way of running software
  businesses.  The message was: there's a revolution coming to the
  industry -- you're choices are to remain where you are, in the past,
  and resist it or to move forward, to the future, and learn the new
  dance. 

  Someone ought to hand some of those tickets to RH.  The non-free RHN
  and private Enterprise strategy are old ways of doing things.  They
  are quite subject to competitive attack -- it's just a matter of
  time.  They aren't, beyond the short-term, a growth strategy.  RH
  ought to embrace its future rather than having it thrust upon it.

  What RH should do is to free up the RHN technology and turn their
  distros into a fully open source project.  If they won't do those
  things, our time (those of interested in furthering the free
  software movement) is better spent working on free alternatives to
  RHN, and on free distributions and the tools that can be used to
  create and manage them in a decentralized way.  Time spent on, for
  example, Fedora advances none of these goals -- it just spares RH
  some of the cost of making their next Enterprise release to feed
  their RHN service.

  Won't such a change in business model mean that that RH has to face
  more difficult competition?  That third parties can undercut their
  RHN subscription prices?  That they won't be the near-exclusive
  providers of a distribution that's recognized by the corporate
  market.  Well, yes -- but: (a) that kind of competition is
  inevitable; (b) they can compete for customers on other grounds,
  most likely with continued success; (c) these changes will
  strengthen and disambiguate RH's role in the free software
  community; (d) these changes are the real growth path in the market
  for RH's investment -- they're better off winding up with a smaller
  share of a much larger market than a lion's share of a doomed market.

  Regarding (a): The inevitability of that new competition is apparent
  in the relatively slight gap between Debian and Red Hat.  The
  tenuousness of that gap is apparent in a project like UserLinux[1].
  It's just a matter of time: will it be 3, 5, or 10 years before
  there is no remaining advantage whatsoever to buying from RH if they
  stay their current course?

  Regarding (b): RH's potential _strength_ in that fight -- it's most
  precious assets -- are it's global infrastructure for providing
  support, it's established and growing market relationships and
  position of trust, and above all it's acclaimed team of hacker
  employees.  Dropping the barriers like "proprietary RHN" and "0wn3d
  distro" means that RH will have to learn new ways to make the best
  use of their other strengths.  I'll suggest some new ways below.

  Regarding (c): The proposed changes will eliminate the two of the
  more significant ethical objections to RH's practices.  Many
  small-scale (and perhaps some large-scale) entrepreneurs could take
  these changes and run with them.   More people would have less
  problematic reasons to help with RH-managed projects -- they'd be
  just another free software player.  I predict that RH could expect
  to see the quality of their offerings rise while the cost of
  production falls.

  Regarding (d): Nobody can predict the future but isn't it a good bet
  that making the benefits of the less-than-free technology RH has
  invested in more freely available will grow the market for services
  that surround it?   A lot?

  So, RH, free up RHN and open up the Enterprise projects.   But then, 
  what will that leave you?   How else can RH grow as a company other
  than by pumping up the number of Enterprise subscriptions?
  That brings us to:


* Forget the "Platform" Orientation -- Embrace Software as a Dynamic
  Process and Seek Mass Markets -- Software is Content

  RH is commanding some pretty impressive prices, relative to costs,
  for enterprise subscriptions.  In their last quarterly report,
  Enterprise subscription revenues grew nearly 160% compared to a
  nearly 20% growth in costs.  Applying the rule of thumb of 200K per
  employee plus a little extra, the enterprise subscriptions alone
  (not their only source of revenue -- just the most interesting one)
  implies a roughly 300 person company.  They were at about 33,000
  subscriptions (that's "installed systems") in that report --
  averaging a price per of just a bit under $2k/yr.

  So what's their plan?  Being a publicly held company, growth is
  their priority.  Can they, over the next 3, 5, 10 years grow the
  number of scriptions from 33K to 3.3M?  Can they be convincingly far
  enough along that path to to make for an attractive target of
  acquisition?

  If you look _only_ at their financial performance over the years and
  interpolate from that:  no sweat.  They haven't run the company
  foolishly from that perspective.

  But they face some serious obstacles and the one most relevent here,
  as expressed in their most recent SEC 10-Q filing is:


     * There are few technology barriers to entry in the open source market. 

     One of the characteristics of open source software is that anyone
     can modify the existing software or develop new software that
     competes with existing open source software. Such competition can
     develop without the degree of overhead and lead time required by
     traditional proprietary software companies. It is possible for a
     competitor with greater resources than ours to develop its own
     open source operating system solution, potentially reducing the
     demand for our solutions.

           -- 
http://eol.finsys.com/\edgar_conv_html\2004\01\06\0001193125-04-000751.html#D10Q_HTM_TX85486_7

  If anything, that risk assessment is understated -- or misleadingly
  stated.  It says: "a competitor with greater resources than ours" --
  who might that be?  Oh, sure -- perhaps Novell will pull something
  out of their butt.  Or Sun.  But I think the real competitor of
  concern ought to be the community at large.
  http://whiteboxlinux.org, UserLinux, http://www.nrh-up2date.org --
  none of these have to succeed bigtime to prove the larger point: my
  bet is that 5-10 years from now, the Enterprise distro/RHN business
  model will be dead.  Not 10x larger.  Dead. [2]

  What, by that time, can they be selling?  If not by endlessly
  raising the number of subscriptions to Enterprise/RHN, how can they
  can hey hope to keep revenues climbing rapidly relative to costs?

  RH's infrastructure for support is, I presume, valuable to
  customers.   Clearly, even if Enterprise/RHN is opened up, at least
  some customers will still want to pay into it.   That's great -- but
  support infrastructure notoriously has costs that grow linearly with
  usage.  They can use this infrastructure to achieve growth of a sort
  -- but not the kind of superlinear growth that investors like.

  One of RH's great strengths, by nearly all reports, is their
  technical staff.   RH is good at cherry-picking great hackers.
  (That they've missed me is, no doubt, a mere oversight :-)

  The traditional view is that technical software development staff
  in the free software world functions economically much like support
  staff and infrastructure:   sure, you can get them gigs for custom
  development, but in doing so your costs always rise in proportion to
  the revenues you can collect -- you're doomed to linear growth.

  I say: bunk to that.  That's bullshit.  Here's why:

  The history of computer programming is a story of explosive growth
  in the efficiency of using human labor.  To put it crudely -- it's a
  popular rule of thumb that, for all time, a given programmer can
  only produce a fixed number of lines of production-quality code per
  day (say, 100/day for a very good programmer, 10/day for an average
  programmer).  That's a universal constant.  What isn't constant is
  "What can be done with just 100 lines of code?"

  100 lines of PDP-11 assembler code from 1978 might implimement, say,
  a matrix operation for an APL implementation.   100 lines of Scheme
  from 1997, on the other hand, might implement a new domain-specific
  programming language in which a useful packet filter can be coded up
  in 10-lines more.  1000 lines of 1983 Pascal code might implement an
  RSX/11 program to read files from a unix tar tape.  1000 codes of
  Emacs lisp code from 2001 might implement an iconic directory viewer 
  that shows me the thumbnails of album covers corresponding to my MP3
  collection and let's me queue up a bunch of them for downloading to 
  my player or playing at my desktop.

  The productivity progress in software is not automatic or free.
  It's a function of software architecture.  For example, 500 lines in
  1983 to implement a particular interface gadget in Motif comes out
  to about 500 lines in 2004 to implement it in GTk.  The 2004 version
  may have advantages like greater customizability of appearence or
  support for localization --- even so, this is an example of an
  architecture that _hasn't_ seen an explosive productivity gain over
  the years.

  So the first observation is that programmer productivity _can_ grow
  non-linearly and we ought to be asking (a) how can we reliably
  reproduce the conditions that cause that explosive growth and (b)
  how we can translate the productivity gain into a corresponding
  revenue gain.

  Next: the Internet really and truly has changed commerce and in some
  ways that are quite compatible with free software.  Two examples
  stand out in my mind: (a) music markets such as iTunes; (b) software
  markets such as the (now less active but once vibrant) market for
  $10 Palm Pilot applications.  What both of these demonstrate is that
  if the price is low enough, people will shop for content if
  presented in a convenient form even when their obligation to pay for
  content is undermined by parallel free distribution.  The Palm
  example demonstrates that the principle applies to software.  We can
  also reason from first principles that people would rather pay a
  little to get software from a trusted source than download it
  randomly.

  At such low prices -- when is a program valuable?   If it provides a
  little entertainment, perhaps?   If it implements an incremental
  convenience?

  Let's put these ideas together:

  Isn't it easy to imagine a future in which:

  ~ nice, little "human scale" applications are cheap to develop

  ~ the most efficient way to deploy such an application on 10,000
    corporate desktops is to send an email to each user, containing
    a URL for a trusted distributor, where clicking on the link 
    both downloads and installs the tiny app _and_ charges the 
    corporate account $.50/pop

  And the idea generalizes:  you might have "cheap new apps" targeted 
  at, say web designers;   others for home users looking for games;
  others for home users looking for ways to manage downloaded music;
  oterhs targetted for people living in a particular geographic or
  political region; etc.  

  Strategically, these observations add up to to three key points for
  the short term:

        1. The long term path to growth comes from maximizing the
           number of users interacting with free software environments
           in such a way that they may wish to install a new tool --
           not from maximizing the number of CPUs running free
           software.  In particular, while the server strategy has
           short and long-term tactical benefit, long-term growth for
           RH is most likely to correlate with the number of end-users
           rather than the number of web pages processed or database
           queries answered.

        2. Software architecture matters and extensible applications
           are key.  Whenever there is a choice between investing in a
           labor intensive architecture that exists (or nearly exists)
           today and investing in a costlier-to-initiate by
           higher-productivity-in-the-long-run extensible
           architecture, a strategic portion of the investment should
           be directed to the extensible architecture.  As a matter of
           strategic R&D, RH should be exploring extensible
           architectures as a primary focus.

        3. End-users are best understood in demographic terms.  
           RH should take a cue from media content developers and
           begin to understand end-users as orangized into demographic
           categories, each of which can be targetted for
           "human-scale" applications.   They should begin to explore
           marketing strategies that help people to identify with
           these communities in the form of participation in on-line
           communities which can later serve as marketplaces for
           human-scale applications.


  To sum it up: long term explosive growth for RH and others will come
  from a shift from software seen as a static or slowly evolving
  commodity to software seen as demographically targetted, rapidly and
  inexpensively produced streams of content.   The "pay-per-view"
  model of cable television, rather than the subscription model of
  Enterprise/RHN is where to look for growth.   This has implications
  today for the kinds of software we invest in and the kinds of market
  niches we seek out.


  -t 

------


[1] The significance of UserLinux

  I have heard people raise some valid complaints about the direction 
  UserLinux is taking and I don't endorse it as a project.

  What I admire about it and find significant is that Bruce has proven
  the existence of members of both the ISV and key-customer
  intellegencia who are actively looking beyond and willing to invest
  in free software solutions that eliminate the lock-in aspects of
  RH's model.


[2] RH's current growth strategy

  As nearly as I can tell, RH's current growth strategy has three main
  parts: 

   1) Seek out new niches, form any necessary ISV parternships, 
      round out free software offerings, and target marketing

      This, it seems to me, is RH's number one strategic strength.
      It's very formulaic:

        ~ identify a niche where most users are running 100% non-free
          systems, preferably on expensive hardware

        ~ design an Enterprised-based solution -- substituting as many
          free software components as possible;  perhaps rounding out
          some free software projects to make them suitable;  seeking
          ISV support for missing components

        ~ sell into that niche, emphasizing the cost savings.


     I predict (and this is hardly a radical prediction) that that
     strategy will continue to work well for at least 3 years.  The
     only serious challenge to it is the maturation of alternatives to
     RHN and Enterprise accompanied by a mixture of niche-targetted
     software projects by free software entrepreneurs/consultants and
     ISV recognition of platforms more open than Enterprise/RHN.  Such
     challenges are inevitable and underway -- but won't be ready for
     prime time by tomorrow.

   2) Acquisitions

     A variation on the niche strategy: RH has a relatively good cash
     position and can acquire companies that have already discovered
     niches of their own.  To the degree RH can succeed at integrating
     those companies with the Enterprise/RHN model (perhaps achieving
     efficiencies the acquired companies could not on their own), they
     can grow through acquisitions.

     I predict that this, too, will continue to work until more open
     alternatives to Enterprise/RHN reach maturity.   At that time,
     companies that might have once been good acquisition targets will
     be more often serving their niches using an already more
     efficient model.   If RH is, at that time, still dedicated to
     Enterprise/RHN in its current form, they will not be able to
     integrate these targets successfully and likely won't be able to
     afford them anyway.

   3) The Roadmap

     RH has a vision for the future:

       http://www.redhat.com/software/architecture/forward.html

     emphasizing their current strengths in infrastructure management
     and seeking to expand into a "pluggable" architecture providing a
     unified management interface for security, storage, and so forth.

     I predict that this vision will achieve only limited realization
     in practice and that it will not provide much that is of unique
     value to RH.   It faces the usual difficulty of all "grand
     unification" architectures -- that of getting applications and
     ISVs to adopt it.   It faces an added difficulty that is
     compounded by free software: people will tend to adopt the
     aspects that are standardized and freely available -- RH will
     have little opportunity to differentiate themselves on the basis
     of that architecture.