protocol idea - could change things considerably :)

[Dan Stromberg]

Teaser question:

How can an automatic build system automatically
determine what is the latest version of a software package, where to find
it, how to download it, how to build it, and what is its relationship to
other pieces of software?


The article:

There exist software systems, that will take a .tar.bz2 or whatever, apply
some patches, run the configure script (for EG), run the Makefile, and
install the resulting binaries.  I'm the author of a little-known one.

Aside from the tweak-at-a-time resolutions to build snafus, there is one
remaining piece of the software build puzzle that is not automated.  But
it could -be- automated, and it wouldn't be that difficult.  I believe the
payoff to automating this last piece, is substantial.

That piece, is a client-server protocol, that allows software
authors/maintainers/vendors to advertise, in a machine-readable form:

1) What is the latest stable version of their software?

2) What is the latest development version of their software?

3) What is the URL to download the latest stable version?

4) What is the URL to download the latest development version?

5) Is the download source or binary?

6) What architecture were the binaries compiled for?

7) What architectures have the sources been built on recently and
successfully?

8) What architectures have the sources been built on recently and
-un-successfully?

9) What other packages does the software download require to
build?

10) What other packages does the software download require to run?

11) What other packages does the software download conflict with?

12) What other packages is the download an alternative to?

This software system could be layered overtop of .tar.Z, .tar.gz,
.tar.bz2, .rpm, .deb, sun pkg's, AIX lpp's, .srpm, and so on.  There's
really not much limit to what kinds of software bundles this could be
applied to.

Such a software system would open the door to "meta distributions", or
distributions that one can create automatically, simply by specifying a
few core programs you require (or a large list for an u"ber distribution),
and letting a build system, layered overtop of the aforementioned
protocol, sort out what else is required or contraindicated, and build
a working *ix system (or other OSes as well) for a given set of
architectures, if feasible, as determined by fairly simple rules
applied to the available data on the internet.

At first, there would probably be a need for a (forgiving) HTML parser, as
well as something that knows how to prowl ftp sites; these tools would
know how to derive as much of the above listed information as
possible, using only traditional protocols: screen scraping as
much of this information as possible from HTML, or doing directory
listings of ftp sites.

Naturally, there would be benefit to having a system that knows how to
take two version numbers and a "version number type" as input, and output
which version number is more recent, and which is older.  EG: What is more
recent?  1.10, or 1.9?  A version number type could distinguish between
the various forms of package numbering in common use.

Later, if the software system is successful, there would be a wave of
authors/maintainers/vendors providing machine-readable information via
the dedicated protocol.

The 12 points listed above, needn't be specified in a black-and-white
fashion.  For example, if "conflict" were described by a percentage, that
would produce a more flexible system (witness the air conditioner that
can produce cooling at a level intended to produce a steady-state, much
like ntp uses, rather than an air conditioner that just has "on" and
"off" modes, and is constantly oscillating between two "high" and "low"
thresholds.  The continuous AC is more energy efficient than the Black
and White AC).  Another possibility is to have a external program
interface that can inspect various variables, and determine if a conflict
exists.  This could probably be applied to most, if not all, of the 12
points above.

If I may be extremely immodest for a while, I'll mention that I believe
that this could change the free software, -and- lock-in software worlds
substantially, and as such, is suitable for a patent application.

I have little interest in restricting access to such a general method, nor
in making minibucks from it (sic), when I believe this could be a "rising
tide lifts all boats" kind of thing for the software industry.

What I -do- have an interest in, is seeing FLOSS software thrive - and
being aware of the patent threat to FLOSS software, I would be -immensely-
pleased if the FSF or OSI or other similar organization, were to patent,
or jointly patent, this software development method, and use it to engage
in cross licensing of patents with lock-in software vendors, for the
benefit of all FLOSS users.

Please let me know if I have hit upon a suitable forum for discussion of
this idea, as well as whether this is a suitable forum for discussing
turning this into a patent.

Thanks for all the great software!