[Axiom-developer] out of source builds

[daly]

I'm not sure why out of source builds are particularly important. But
there are two points to discuss.

The first point, mentioned in a prior email, is that there is a 
general class of requirements:
  (b) doing it because that's the way everybody does it
and this seems to be of that character. If there is a different
need anyone can point to as a good argument why this is important? I 
understand the CD argument as I put Axiom on the ISSAC CDs.
However this is a rare situation for distributing any open source
software these days. I cannot remember getting sources for an open
source project on CD. 

(Caveat: I had to recreate the whole build subsystem from scratch
so some of these features are not fully tested but they are there
by design. In particular, the newly added literate facilities are
not correct yet (e.g. Makefiles are extracted to "src" instead of
"obj/sys", which is a bug that needs to be fixed))

The second point is about the claim that the current build system
cannot build "out of source".  By design the "src" tree should never
be touched by Make so you should be able to mount the source read-only
and type Make. Any failures are a bug.

The current build system IS designed to do out of source builds. To do
this you need to notice two things. First, the top level Makefile
contains the lines:
  SRC=${SPD}/src
  INT=${SPD}/int
  OBJ=${SPD}/obj
  MNT=${SPD}/mnt

Second, notice that all of the file manipulations in the Makefiles
are prefixed by paths built from these 4 variables. Thus, to do
out of source builds it should suffice to just change these:
  SRC=${SPD}/src
  INT=${MYHOME}/int
  OBJ=${MYHOME}/obj
  MNT=${MYHOME}/mnt

All other Makefiles create IN, MID, OUT, DOC, etc from these. For
example see src/interp/Makefile. At the top you'll see:
  IN=${SRC}/interp
  MID=${INT}/interp
  OUT=${OBJ}/${SYS}/interp
  DOC=${MNT}/${SYS}/doc/src/interp
and the SYS variable is gotten from `basename $AXIOM` which is
a name to distinguish the type of target to build.






There were other issues back in the scratchpad days. Since a build 
could take weeks it was vital to architect the build system so the
  (1) builds need to be done minimizing the re-work.
  (2) builds could be done in parallel, out of source, multiple targets
  (3) builds only built the request parts
The current build system is designed to do all three.

MINIMIZE REWORK

Requirement (1), that builds need to be done minimizing the re-work,
is accomplished by the "int" subdirectory. Because Axiom uses boot
there is a whole translation stage needed to extract the source code.
The "src" source tree gets extracted into the "int" source tree. Many
steps, such as the Boot->Lisp translation are system independent and
only need ever be done once. Unless the boot code is changed there
is no reason to redo the work and waste the time, which is the point
of using Makefiles at all.

The "int" directory represents an Axiom build that considerably
speeds up the build process. The "int" directory
  (1) caches the Boot->Lisp translation (removes the need for 
      running bootsys, thus removing a whole build "pass".)
  (2) caches the .fn files for code optimization (allows optimal
      code to be generated on second compiles which improves the
      execution time of the final system)
  (3) caches the algebra system-independent translations
      (the .nrlibs contain several files including .lsp files
       which are post-algebra compiles. since algebra translation
       is an expensive step this is a huge savings)
  (4) caches the pamphlet weave output and tangle output (removes
      the need to run noweb at all)
  (5) caches the latex aux files so latex need only run once rather
      than twice for every file. (halfs latex time)

In theory you could ship the "int" directory and do a complete build.
With proper caching this could reduce the whole build time considerably,
probably to less than half an hour.

There has been much discussion about how long builds take and this is
a real concern. So careful attention to the system-independent cache
would create a huge savings of time, reducing single-file changes to
just a minute or so.

Make needs to pay close attention to doing rework only if a source
file changes. Otherwise Make can use the cached information in "int"
and reduce the build times by large factors (e.g. no algebra compiles).

I doubt anyone notices this feature because they probably do
   make clean
which destroys the cache. However Axiom is designed to do incremental
changes. You should be able to modify any source file, type 'make'
and have the system only rebuild the parts that are affected. It used
to do this and is designed to do this. Thus the issue about long
system builds should only happen on new systems. If you can't minimize
rework what is the point of using Makefiles?

After the first time you should almost never need to rebuild Axiom
from scratch, at least by design of the current build system. We
rarely did in the scratchpad project because it took weeks.






PARALLEL BUILDS, OUT OF SOURCE, MULTIPLE TARGETS

Requirement (2), that builds be done in parallel, was needed so that
multiple machines could work on the same source tree. Since Scratchpad
was build on wildly different architectures (Symbolics, IBM/RS6000,
IBM PC, SUN, VM/370) and wildly different lisps (Zetalisp, CMUCL,
AKCL, Golden Common Lisp, etc) there was a lot of build time involved.

The solution to this problem was the "obj" directory and NFS. By
design, if you do a build on any one system you will do 2 things:
 (1) you will cache all the rework into the "int" directory
 (2) you will create a new directory (e.g. obj/SUN, obj/RIOS)
     for that particular target system.

Once you have a valid system build life is good. All you have to
do on a new target system, e.g. PC386, is:
  make a new directory, 
      mkdir AX
  use NFS to network mount the old src and int directories readonly
      mount -readonly NFS:/home/daly/AX/src AX/src
      mount -readonly NFS:/home/daly/AX/int AX/int
  set the AXIOM variable
      export AXIOM=AX/mnt/PC386
  and run make
      make

Since the build system knows about int and the int files are newer
than src the build on the new system, PC386, only needs to compile
the lisp files for the interpreter and algebra. It only needs to 
compile the C files for graphics and hyperdoc. All of this work went
into the "obj/sys" directories. If the master source changed all you
had to do on the other systems was simple rebuilds.

This would be very useful if and when we have a compile-farm setup
with many different target architectures and different lisp systems.
All you need to do is a network mount and everybody builds from the
same src/int tree.





BUILD ONLY REQUESTED PARTS

The requirement (3) build only the request parts is partially
re-implemented but not fully. Look at the top level Makefile for
the lines:
   PART= cprogs
   SUBPART= everything

This information is passed in the ENV variable. Using this the
lower level Makefiles can skip whole subdirectories. You used to
be able to specify that you only wanted to build hyperdoc with:
   make PART=hyperdoc
and you could specify subparts, e.g. the hyperdoc pages
   make PART=hyperdoc SUBPART=pages






When I recreated the build system (post NAG) I was not exceptionally
careful to ensure that the "int" directory and "obj" directories
where especially "pure" but that is a trivial fix. It only requires
prefixing the output filenames in the makefiles with the right paths.
This is maybe a day's worth of work. Recovering the "PART" ability
is also reasonably simple since it only requires checks in Makefiles
before calling Make on subdirectories.

Axiom has been built for years and no-one has mentioned out of source
builds before. If this is a real issue then the current system is
designed to do it (modulo Makefile mistakes).

Tim