Re: Axiom & internal/external GCL (was: Re: [Axiom-developer]Axiomcvs 040624 build error)

[root]

> > I think, we should unbundle gcl and other additional
> > software. To get Axiom running and some gcl interactions 
> > resolved, it was certainly ok to pack everything together in 
> > the beginning. But these problems should be solved now. Most 
> > package systems (debian, BSDs etc) expect that the lisp 
> > system and other packages are separate. Besides it will
> > make Axiom much easier to port to other platforms. The
> > technical details of loading C code have already been
> > adressed in Camm's mails. 
> 
> I agree with this. It is normal practice for complex systems
> like Axiom to specify dependencies. After all, GCL depends
> intimately on GCC and binutils but we do not include a copy
> of these in the Axiom installation. What is needed is simply
> to document the known dependencies and to modify the Axiom
> build to make patching GCL (and noweb) unnecessary.
> 
> Regarding ports to other systems, we are already doing this
> in the case of the port to Windows.

Well, as they say, advocacy is volunteering. :-)

If we were to attack the problem of making Axiom live without 
specific GCL dependencies I'd suggest that we need to broaden
the base of lisps that it runs on. The lisp code in the interpreter
already contains #+ and #- for some half-dozen lisp systems (I used
to have it running on spice (cmucl), allegro, symbolics, vmlisp, etc.
It also was released to me running on CCL by Arthur Norman. In fact,
I've continued to include CCL with the specific intention of getting
that code running again. 

The top level makefile contains parameters for customizing the
system to other lisps (e.g. the name of the extensions vary among
the lisps (.clisp, .lisp, .lsp, .l, etc for source, .o, .x86, .obj, etc
for binaries). These can easily be customized by simple one-line
changes. (Which will, of course, break things where I was forgetful
but the design and most of the implementation exists).

The plan is to build on other lisps. CMUCL is an important one
because it has a lot of nice optimization tools that were used
to improve the lisp code. CCL is important because it allows the
system to be quickly ported (it is a byte-coded lisp similar to
the java runtime). 

The issue is not intention but time and knowledge.

It is true that Camm knows and has written down the technique for
dynamically loading C code with GCL lisp cover functions. I've seen it
but not internalized it. However, in order to "do it right" we need
to find the same functions in at least 2 other lisps (e.g. CMUCL
and CCL) or if they don't exist to implement those techniques. Then
we need to create new files that #+ and #- the various lisp-level
enhancements so they work in a transparent way. And then we need
to add a layer to the build process so that the code is properly
compiled at the right time (likely between the lisp and boot steps).

Someone needs to build a GCL image (the code is there already)
and explain how to dynamically link to C code with lisp cover 
functions.

Someone needs to build a CCL image (the code is there already)
and explain how to dynamically link to C code with lisp cover 
functions.

Someone needs to build a CMUCL image (the code used to be there)
and explain how to dynamically link to C code with lisp cover 
functions.

Given these three examples we can design a generic layer that
handles all three. It should then be a simple job to make it
fully portable and we can stop shipping any lisp CVS information.


My general plan (at the moment) lies along the following lines:

current state
  rebuild
    graphics capability  (1st draft just uploaded)
    numeric capability   (1st draft in process locally)
    hypertex capapility  (1st draft in process locally)
    GUI capability       (ltk?)
  document
    complete original axiom book (1st draft uploaded)
    complete 'gemstone' book based on littlewood (on private website)
    literate framework to handle all pamphlets/booklets (advi rewrite)
    literate version of algebra (outstanding requests for papers)
    literate version of internals (ongoing)
  testing
    re-automate regression testing 
    CATS (computer algebra test suite) documented test cases (locally)
  platform port
    windows (Camm, Bill, etc)
    apple   (BSD tested but failed)
  lisp port
    ccl     (partial documentation)
    cmucl   
    clisp

within each of these areas (rebuild, document, testing, platform port,
and lisp port) are the mundane tasks, the "up to date and innovate"
tasks and the research tasks.

For example, in the algebra area we need to rebuild the numerics, 
document the existing code and re-automate the regression tests (the
machinery is there but not yet recovered). That is all mundane work.
For "up to date and innovate" in the algebra we need to bring Axiom
in line with the state of the art. For instance, we need to have the
capability to do symbolic summation. For the research work, there is
the push to integrate ACL2 (a proof language), the indefinite research
recently discussed here, the push to do more symbolic manipulation
(i.e. solve problems by theorems rather than computation), develop
the GUI to be able to derive an "intensional stance", etc.

Axiom is a reasonably large project and I'm very open to suggestion
about development. The fact that it ships with GCL sources is not
really by design but is due to my ignorance about a general mechanism.
If anyone designs a general mechanism that works in GCL, CCL, and
CMUCL we can rewrite the whole lower layer to use it and elide the GCL
sources. I know it can be done, I just haven't had the time yet.

Tim