RE: [Axiom-developer] letting my mud settle

[Bill Page]

On November 7, 2005 10:20 PM Tim Daly wrote:
>...
> the task at that time was to rewrite the system into pamphlet
> files. had i proposed such a massive, pervasive rewrite this
> mailing list would have been aflame with opinion. and the
> consensus would certainly have been to NOT do a pamphlet
> rewrite or if so, use doxygen, texmacs, xml, etc.

I think Tim is wrong about this. I do not recall any objection
at all to the concept of using literate programming methods in
Axiom. I still do not hear any objections to the use of "pamphlet"
files. For the most part this change was just syntactical, i.e.
wrapping existing source code into a literate programming
container. That change was dealt with by expanding the make
scripts.

The only thing that I think was broken by this was that at the
same time as packaging stuff into pamphlet files, Tim also
decided to group the algebra source code modules (about 1,300
files) into a smaller number of pamphlet files (about 300). This
was not a bad idea in principle but it broke the link to the
source files what was built into the Axiom )show command and
into HyperDoc. See Issue #144

http://wiki.axiom-developer.org/144DomainAbbreviationIsNoLongerAssociatedWit
hFilename

because the relationship between pamphlet files and the Axiom
user interface had not been fully thought out. Of course this
can be fixed but it requires some thinking about how we want
this part of the Axiom user interface to work. Does it still
make sense to access Axiom algebra source files this way given
that pamphlet files mix algebra code and it's documentation in
raw LaTeX format? Perhaps a link to the dvi and/or pdf file
would be more appropriate, with a secondary link to the pamphlet
source file, e.g. like the pamphlet support on MathAction?

> 
> in the process of building the system and the Makefile tree
> i spent a lot of time thinking about my past sins (and
> suffering from them which is poetic justice). as a result
> of that effort i decided that the system needed to be
> completely rewritten.

I am not aware of any of Tim's past sins -- only his present
ones. ;) Building Axiom from sources only, which was apparently
a requirement imposed by restrictive licensing conditions which
apparently prevented any of the previously commercial binary
versions of Axiom to be distributed along with the Axiom source
code, was certainly a challenge because of the way that Axiom
had been designed to be "bootstrapped" from an existing running
copy. But this is no different than the situation with most
compilers and in particular GNU C (gcc).

I think that William Sit is right that having accomplished this
bootstrap, it seems unnecessary to continue to distribute Axiom
in this way. It is now quite possible to return to the way Axiom
used to be distributed as binary code plus source code. Building
open source Axiom would then require a running version of open
source Axiom in exactly the same way that building gcc from
source code requires a running gcc.

Since this thread is at least partly devoted to "airing dirty
laundry", I should mention that in retrospect I think the specific
way in which Axiom algebra was bootstrapped from source, i.e. by
including machine generated lisp code from Spad compiler output
directly in the pamphlet files was probably not the best idea.
At the time Tim made this decision I was not really part of the
project and even if I had been I would not have understood then
that there was a better way. But now it is clear to me that the
cyclic dependencies in Axiom's library code are a result of the
fairly extensive use of mutual recursion, i.e. source modules
that naturally recursively depend on each other. This may not
have been especially obvious even to the original Axiom developers
since Axiom had always previously been built from an existing
running version of Axiom and the algebra code as written over a
fairly long period of time.

In theory the better way, instead of patching in the lisp output
of a previous version of Axiom, would have been (and which still
may be) to compile Spad "stub" files consisting only of the initial
code with all dependencies removed, for those Spad files which
had been specially selected to break the dependency cycles - or
for that matter, even compiling all existing Spad files - first
as "stubs' and then a second time in their full source form.

We have previously discussed this extensively as part of the
thread:

http://wiki.axiom-developer.org/17AlgebraBOOTSTRAPFixedPoint
http://wiki.axiom-developer.org/MutualRecursion

I think that recognizing this is important because it is
really a **feature** of the Spad language and the Axiom algebra
library in particular. It is not a *problem* as such, but rather
a challenge for the compiler developers. Aldor at least partially
addresses this through it's "extend" mechanism.

> the task at this time is to document it and clean it up.

I think as Tim will admit, that task is still at it's beginning
stage. Very few (maybe less than 10%) of the Axiom source code
pamphlet files currently contain any useful documentation besides
the comments contained in the original code. And even to get this
far Tim has had to devote a lot of time and effort to just creating
the amount of documentation that does exist and is essential to
building Axiom from the current source distribution - especially
in the make files.

> i need to get to the point of being able to say 
> 
>   (compile-file "interpreter.lisp") ; actually bookvol5
>   (load "interpreter")
>   (compile-file "compiler.lisp")    ; actually bookvol6
>   (load "compiler")
>   (compile-file "browser.lisp")     ; actually bookvol7
>   (load "browser")
>   (compile-file "graphics.lisp")    ; actually bookvol8
>   (load "graphics")
>   (axiom)
> 

To me this vision of how to build Axiom seems terribly myopic.
I don't think it addresses the really important issues about
Axiom's design. And more over, getting a stage where something
like the above build sequence might be possible is likely to
both distract from the all important task of documentation plus
lead us away from some of the things that really need to be
documented most, e.g. the relationship between Spad, Boot and
Lisp.

> at that time we'll have a crisp, clean, documented, partitioned 
> system and we will be able to understand the parts. we can
> rewrite them in aldor if someone steps up to it. or add new ones.
> for example
> 
>   (compile-file "openmath-server.lisp") ; bookvol9?
>   (load "openmath-server")
>   (axiom :openmath-port 1234)
>

> we can't do that now. and we can't do it in boot. the system is
> too damn complex and badly designed/written/structured. construction
> is always dirty, disruptive, and time consuming.

Even with deficient documentation, I don't see anything that prevents
us from doing this sort of thing right now. I think Kai Kaminski's
work on AxiomUI is a good demonstration that this is indeed possible.

> i'm sorry that you'll get to see my ugly hacking along the way.
> but, hey, that's open source programming.

I don't think anyone here (except perhaps you, Tim) is worried about
seeing "ugly hacking". All of that is an expected part of any large
programming project. What is much more important are the conceptual
issues and the overall approach. Is your approach one that will
encourage other developers to help with the task or is it one that
makes it likely that only for you to carry-out all the work?

> 
> i'm going to fix axiom.
> i know how to get there from here. 
> it takes time. 
> it takes work.
> it takes patience.
>

As you say: "that's open source programming", but I don't think
that it must be this way.
 
> and once this is done there is SO much more to do.
> why, the algebra restructuring alone will take years...
> 

I agree that Axiom's 30 year horizon is not unrealistic except
for the human physical limitations of lifespan (and attention
span :). I think that if Axiom is really going to continue to
live passed the span of it's original developers, then we must
plan very carefully to attract and keep a new generation of
developers. This is the "bottom line" that motivates me to
continue this rather difficult thread of discussion.

Regards,
Bill Page.