[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Axiom-developer] Debian/Stable Packages Built + Summary of issues
From: |
Camm Maguire |
Subject: |
Re: [Axiom-developer] Debian/Stable Packages Built + Summary of issues |
Date: |
29 Mar 2005 10:34:34 -0500 |
User-agent: |
Gnus/5.09 (Gnus v5.9.0) Emacs/21.2 |
Greetings!
root <address@hidden> writes:
> Camm,
>
> in src/interp/Makefile.pamphlet in the axiom--BSD branch the attached
> two blocks of code exist. I need to move them out of the Makefile into
> a lisp file, call them with reasonable parameters, make them load-time
> conditional blocks based on GCL, write non-GCL versiosn and document why
> they are needed and what they do. This is rather hard as I'm not even
> sure what the problem is :-)
>
Indeed! While I'm not sure of the specific requirements on BSD, I am
guessing that they went with this code to be uniform across many
platforms which presumably include at least some of ia64 mips hppa and
alpha. In short, this code below is not needed for other
architectures, whether on Linux of xBSD.
There are two issues as mentioned earlier:
1) build with external GCL. Debian and xBSD presumably prefer this
for modularity.
2) avoid si::save-system as presently required only on ia64 mips hppa
and alpha, but works elsewhere too. Debian only does this where
required, xBSD appears to be doing it everywhere. si::save-system
is not only easier on build resources, much easier to use, and much
more faithful to the spirit of lisp, but it is also what GCL would
like to extend going forward, though this will doubtlessly take
time. The alternative, compiler::link, makes the image via ld,
which of course is much more widely tested and ported, but requires
that the heap be regenerated from scratch on each image save.
I'll repost here the three patches we use in Debian, one of which
includes the below, with comments:
=============================================================================
patch.all
=============================================================================
--- ./src/graph/view3D/globals.h.orig 2005-01-04 23:58:20.000000000 +0000
+++ ./src/graph/view3D/globals.h 2005-02-14 18:30:37.000000000 +0000
@@ -132,6 +132,8 @@
extern char propertyName[];
extern char propertyBuffer[];
+#undef R1
+#define R1 RR1
extern float transform[4][4], transform1[4][4],
R[4][4], R1[4][4], S[4][4], T[4][4], I[4][4];
extern float vxmax,vxmin,vymax,vymin,
*******************************************************
R1 used internally on arm:
*******************************************************
--- ./src/algebra/Makefile.pamphlet.orig 2005-01-30 12:03:09.000000000
+0000
+++ ./src/algebra/Makefile.pamphlet 2005-02-15 00:01:26.000000000 +0000
@@ -1598,7 +1598,7 @@
@ echo generic 3 -- making ${MID}/$*.o from ${MID}/$*.lsp
@ (cd ${MID} ; \
echo '(progn (in-package (quote boot)) (compile-file "$*.lsp"
:output-file "$*.o"))' | ${DEPSYS} )
- @ cp ${MID}/$*.o ${OUT}/$*.o
+ @ mkdir -p $$(dirname $(OUT)/$*.o) && cp ${MID}/$*.o ${OUT}/$*.o
@
<<genericSPADfiles>>=
*******************************************************
Had trouble with directories being created
*******************************************************
=============================================================================
patch.nosave (ia36 m68k arm ppc sparc s390 amd64)
=============================================================================
--- ./lsp/Makefile.pamphlet.orig 2005-01-30 12:03:08.000000000 +0000
+++ ./lsp/Makefile.pamphlet 2005-02-14 18:34:30.000000000 +0000
@@ -706,15 +706,7 @@
@echo 1 building ${LSP} ${GCLVERSION}
gcldir:
- @echo 2 building ${GCLVERSION}
- @tar -zxf ${ZIPS}/${GCLVERSION}.tgz
-<<gcl-2.6.5.socket.patch>>
-<<gcl-2.6.5.libspad.patch>>
-<<gcl-2.6.5.toploop.patch>>
-<<gcl-2.6.5.h.gmp_wrappers.h.patch>>
-<<gcl-2.6.5.tail-recursive.patch>>
-<<gcl-2.6.5.collectfn.fix>>
-<<gclConfigureMake>>
+ echo '(compiler::link nil "${OUT}/lisp" (format nil "(progn (let
((*load-path* (cons ~S *load-path*))) (compiler::emit-fn t))(when (fboundp
(quote si::sgc-on)) (si::sgc-on t)))" si::*system-directory*)
"${OBJ}/${SYS}/lib/cfuns-c.o ${OBJ}/${SYS}/lib/sockio-c.o
${OBJ}/${SYS}/lib/libspad.a")' | gcl
@echo 13 finished system build on `date` | tee >gcldir
*******************************************************
This is basically equivalent to your manipulation of the EXTRAS
makefile variable in the local gcl build tree, which of course is not
available when using GCL externally. Here is the code expanded:
(compiler::link ;; produce a new image using ld
nil ;; including no new compiled lisp modules
"${OUT}/lisp" ;; to this path
(format nil ;; Running this command in the new image to
;; recreate the heap
"(progn
(let ((*load-path* (cons ~S *load-path*))) ;;
autoload gcl_collectfn and turn
(compiler::emit-fn t)) ;; on
function tabulation
(when (fboundp (quote si::sgc-on)) (si::sgc-on t)))"
;; Turn on sgc gbc algorithm where present
si::*system-directory*)
;; Link in these C objects and libraries
"${OBJ}/${SYS}/lib/cfuns-c.o ${OBJ}/${SYS}/lib/sockio-c.o
${OBJ}/${SYS}/lib/libspad.a"
)
*******************************************************
ccldir: ${LSP}/ccl/Makefile
--- ./src/interp/Makefile.pamphlet.orig 2005-01-30 12:04:38.000000000 +0000
+++ ./src/interp/Makefile.pamphlet 2005-02-14 18:37:17.000000000 +0000
@@ -672,8 +672,10 @@
@ echo '#+:akcl (setq compiler::*suppress-compiler-notes* t)' >>
${OUT}/makeint.lisp
@ echo '#+:akcl (si::gbc-time 0)' >> ${OUT}/makeint.lisp
@ echo '#+:akcl (setq si::*system-directory* "${SPAD}/bin/")' >>
${OUT}/makeint.lisp
+# @ (cd ${OBJ}/${SYS}/bin ; \
+# echo '(progn (gbc t) (load "${OUT}/makeint.lisp") (gbc t)
(user::spad-save "${SAVESYS}"))' | ${LISPSYS} )
@ (cd ${OBJ}/${SYS}/bin ; \
- echo '(progn (gbc t) (load "${OUT}/makeint.lisp") (gbc t)
(user::spad-save "${SAVESYS}"))' | ${LISPSYS} )
+ echo '(progn (gbc t) (setq x si::*system-directory*)(load
"${OUT}/makeint.lisp") (setq si::*system-directory* x) (unintern (quote x))(gbc
t)(user::spad-save "${SAVESYS}"))' | ${LISPSYS} )
@ echo 6 ${SAVESYS} created
@ cp ${SAVESYS} ${AXIOMSYS}
@ echo 6a ${AXIOMSYS} created
*******************************************************
This is entirely to preserve the si::*system-directory* variable,
which is needed to find certain files on an extnerally installed GCL
system.
*******************************************************
=============================================================================
patch.save (ia64 alpha mips hppa)
=============================================================================
--- ./lsp/Makefile.pamphlet.orig 2005-01-30 12:03:08.000000000 +0000
+++ ./lsp/Makefile.pamphlet 2005-02-14 18:39:30.000000000 +0000
@@ -706,15 +706,7 @@
@echo 1 building ${LSP} ${GCLVERSION}
gcldir:
- @echo 2 building ${GCLVERSION}
- @tar -zxf ${ZIPS}/${GCLVERSION}.tgz
-<<gcl-2.6.5.socket.patch>>
-<<gcl-2.6.5.libspad.patch>>
-<<gcl-2.6.5.toploop.patch>>
-<<gcl-2.6.5.h.gmp_wrappers.h.patch>>
-<<gcl-2.6.5.tail-recursive.patch>>
-<<gcl-2.6.5.collectfn.fix>>
-<<gclConfigureMake>>
+ echo '(compiler::link nil "${OUT}/lisp" (format nil "(progn (let
((*load-path* (cons ~S *load-path*))(si::*load-types* ~S)) (compiler::emit-fn
t))(when (fboundp (quote si::sgc-on)) (si::sgc-on t))(setq
compiler::*default-system-p* t))" si::*system-directory* (quote (list ".lsp")))
"${OBJ}/${SYS}/lib/cfuns-c.o ${OBJ}/${SYS}/lib/sockio-c.o
${OBJ}/${SYS}/lib/libspad.a")' | gcl
@echo 13 finished system build on `date` | tee >gcldir
ccldir: ${LSP}/ccl/Makefile
*******************************************************
This is the same as the above linking in of libspad.a but
a) turns on compiler::*default-system-p* -- lisp files to be
linked with ld must be compiled with the :system-p t flag
b) loads gcl_collectfn.lsp instead of gcl_collectfn.o. On these
systems, fasloading is via dlopen which cannot be preserved
across image saves. Any binary modules desired in a new image
must go through compiler::link, alas, at present
=============================================================================
--- ./src/algebra/Lattice.pamphlet.orig 2005-01-04 23:45:59.000000000 +0000
+++ ./src/algebra/Lattice.pamphlet 2005-02-14 18:45:10.000000000 +0000
@@ -39620,13 +39620,14 @@
@ cp -p ${SRC}/doc/gloss.text ${LIB}
@ cp -p ${SRC}/doc/topics.data ${MID}
@ echo rebuilding daase files
- @ (cd ${MID} ; \
- echo ')set out le 200' >/tmp/tmp.input ; \
- echo ')fin' >>/tmp/tmp.input ; \
- echo '(make-databases "" (QUOTE ("unix")))' >>/tmp/tmp.input ; \
- echo '(bye)' >>/tmp/tmp.input ; \
- cat /tmp/tmp.input | ${INTERPSYS} ; \
- rm -f /tmp/tmp.input )
+# @ (cd ${MID} ; \
+# echo ')set out le 200' >/tmp/tmp.input ; \
+# echo ')fin' >>/tmp/tmp.input ; \
+# echo '(make-databases "" (QUOTE ("unix")))' >>/tmp/tmp.input ; \
+# echo '(bye)' >>/tmp/tmp.input ; \
+# cat /tmp/tmp.input | ${INTERPSYS} ; \
+# rm -f /tmp/tmp.input )
+ @ (cp ${SRC}/../debian/*.daase ${MID})
@ echo If all went well, go-ahead Mike and do a db-install as well !
db-install:
@@ -39758,7 +39759,8 @@
@ echo rebuilding databases...
@ cp ${SRC}/doc/gloss.text ${MID}
@ cp ${SRC}/doc/topics.data ${MID}
- @ (cd ${MID} ; echo ')lisp (make-databases "" nil)' | ${INTERPSYS} )
+# @ (cd ${MID} ; echo ')lisp (make-databases "" nil)' | ${INTERPSYS} )
+ @ (cp ${SRC}/../debian/*.daase ${MID})
check:
@ echo Checking that INTERP.EXPOSED and NRLIBs are consistent
--- ./src/etc/Makefile.pamphlet.orig 2005-01-30 12:03:12.000000000 +0000
+++ ./src/etc/Makefile.pamphlet 2005-02-14 18:47:16.000000000 +0000
@@ -33,9 +33,10 @@
@ cp ${SRC}/doc/gloss.text ${INT}/algebra
@ cp ${SRC}/doc/topics.data ${INT}/algebra
@ cp ${SRC}/doc/topics.data ${INT}/algebra
- @ (cd ${INT}/algebra ; \
- echo ')lisp (make-databases "" nil)' | ${INTERPSYS} )
- @ cp ${INT}/algebra/*.daase ${MNT}/${SYS}/algebra
+# @ (cd ${INT}/algebra ; \
+# echo ')lisp (make-databases "" nil)' | ${INTERPSYS} )
+# @ cp ${INT}/algebra/*.daase ${MNT}/${SYS}/algebra
+ @ (cp ${SRC}/../debian/*.daase ${MNT}/${SYS}/algebra)
@
\section{summary}
*******************************************************
On these systems using dlopen, only 1024 files can be loaded in a
given session. We cannot rebuild the databases here therefore, so
they are copied from versions generated and saved on an i386 build
*******************************************************
--- ./src/boot/Makefile.pamphlet.orig 2005-01-30 12:03:11.000000000 +0000
+++ ./src/boot/Makefile.pamphlet 2005-02-14 18:50:14.000000000 +0000
@@ -1151,7 +1151,8 @@
expansion. Adding a single quote symbol will break this expansion.
<<environment>>=
-CMD0= (progn (mapcar (function (lambda (x) (load x))) (quote (${OBJS1})))
(system::save-system "${SAVESYS}"))
+#CMD0= (progn (mapcar (function (lambda (x) (load x))) (quote (${OBJS1})))
(system::save-system "${SAVESYS}"))
+CMD0= (compiler::link (quote (${OBJS1})) "${SAVESYS}" (format nil "(let
((*load-path* (cons ~S *load-path*))(si::*load-types* ~S)) (compiler::emit-fn
t)) (when (fboundp (quote si::sgc-on)) (si::sgc-on t)) (setq
compiler::*default-system-p* t)" si::*system-directory* (quote (list ".lsp"))))
@
\subsection{boothdr.lisp \cite{1}}
*******************************************************
Here is the compiler::link version of the boot image creation. It is
the same as the lisp image creation, except that the list of compiled
lisp objects is supplied in the second argument.
*******************************************************
--- ./src/interp/Makefile.pamphlet.orig 2005-01-30 12:04:38.000000000 +0000
+++ ./src/interp/Makefile.pamphlet 2005-02-14 18:53:20.000000000 +0000
@@ -619,8 +619,31 @@
@ echo '(load "${OUT}/c-util")' >> ${OUT}/makedep.lisp
@ echo '(unless (probe-file "${OUT}/g-util.${O}") (compile-file
"${OUT}/g-util.${LISP}" :output-file "${OUT}/g-util.${O}"))' >>
${OUT}/makedep.lisp
@ echo '(load "${OUT}/g-util")' >> ${OUT}/makedep.lisp
+# @ (cd ${MNT}/${SYS}/bin ; \
+# echo '(progn (load "${OUT}/makedep.lisp") (spad-save "${DEPSYS}"))'
| ${LISPSYS})
@ (cd ${MNT}/${SYS}/bin ; \
- echo '(progn (load "${OUT}/makedep.lisp") (spad-save "${DEPSYS}"))'
| ${LISPSYS})
+ echo '(progn \
+ (setq si::*collect-binary-modules* t) \
+ (load "${OUT}/makedep.lisp") \
+ (compiler::link \
+ (remove-duplicates si::*binary-modules* :test (quote
equal)) \
+ "$(DEPSYS)" \
+ (format nil "\
+ (setq si::*collect-binary-modules* t) \
+ (let ((si::*load-path* (cons ~S
si::*load-path*))\
+ (si::*load-types* ~S))\
+ (compiler::emit-fn t))\
+ (load \"$(OUT)/makedep.lisp\")\
+ (gbc t)\
+ (when si::*binary-modules* \
+ (error si::*binary-modules*))\
+ (setq si::collect-binary-modules* nil
si::*binary-modules* nil)\
+ (gbc t)\
+ (when (fboundp (quote si::sgc-on)) (si::sgc-on
t))\
+ (setq compiler::*default-system-p* t)\
+ " si::*system-directory* (quote (list ".lsp")))\
+ "" \
+ nil))' | $(LISPSYS))
@ echo 4 ${DEPSYS} created
*******************************************************
Here is the compiler::link version of the depsys creation, unpacked
and commented:
(progn
(setq si::*collect-binary-modules* t) ;; When this is set,
si::*binary-modules* contains
;; a list of all fasloaded .o files
(load "${OUT}/makedep.lisp") ;; Run the depsys creation stuff
(compiler::link ;; Link an image with ld
(remove-duplicates si::*binary-modules* :test (quote equal)) ;; Supply the
list of all loaded .o files to ld here.
"$(DEPSYS)" ;; Save the image here
;; Run the string below in the saved
image to recreate the heap,
;; redirecting all calls to (load
foo.o) where foo.o is already linked in
;; to a mere initialization of the
module. GCL will report "Initializing ..."
;; instead of "loading ..."
(format nil "
(setq si::*collect-binary-modules* t) ;; Collect loaded .o
files again to check for errors
(let ((si::*load-path* (cons ~S si::*load-path*)) ;; Turn on
function profiling using
(si::*load-types* ~S)) ;;
gcl_collectfn.lsp instead of gcl_collectfn.o
(compiler::emit-fn t))
(load "$(OUT)/makedep.lisp") ;; Run the depsys
creation stuff again, redirecting loads
;; of already linked
files to initializations instead
(gbc t)
(when si::*binary-modules* ;; Make sure we didn't
really load any .o file
(error si::*binary-modules*))
(setq si::collect-binary-modules* nil si::*binary-modules* nil)
;; All the below the same as usual
(gbc t)
(when (fboundp (quote si::sgc-on)) (si::sgc-on t))
(setq compiler::*default-system-p* t)
" si::*system-directory* (quote (list ".lsp")))
""
nil))
*******************************************************
@
@@ -672,8 +695,36 @@
@ echo '#+:akcl (setq compiler::*suppress-compiler-notes* t)' >>
${OUT}/makeint.lisp
@ echo '#+:akcl (si::gbc-time 0)' >> ${OUT}/makeint.lisp
@ echo '#+:akcl (setq si::*system-directory* "${SPAD}/bin/")' >>
${OUT}/makeint.lisp
+# @ (cd ${OBJ}/${SYS}/bin ; \
+# echo '(progn (gbc t) (load "${OUT}/makeint.lisp") (gbc t)
(user::spad-save "${SAVESYS}"))' | ${LISPSYS} )
@ (cd ${OBJ}/${SYS}/bin ; \
- echo '(progn (gbc t) (load "${OUT}/makeint.lisp") (gbc t)
(user::spad-save "${SAVESYS}"))' | ${LISPSYS} )
+ echo '(progn \
+ (setq si::*collect-binary-modules* t)\
+ (setq x si::*system-directory*)\
+ (load "${OUT}/makeint.lisp")\
+ (setq si::*system-directory* x)\
+ (unintern (quote x))\
+ (compiler::link \
+ (remove-duplicates si::*binary-modules* :test
(quote equal))\
+ "$(SAVESYS)" \
+ (format nil "\
+ (let ((si::*load-path* (cons ~S
si::*load-path*))\
+ (si::*load-types* ~S))\
+ (compiler::emit-fn t))\
+ (setq si::*collect-binary-modules* t)\
+ (setq x si::*system-directory*)\
+ (load \"$(OUT)/makeint.lisp\")\
+ (setq si::*system-directory* x)\
+ (unintern (quote x))\
+ (when si::*binary-modules* \
+ (error si::*binary-modules*))\
+ (setq si::collect-binary-modules* nil
si::*binary-modules* nil)\
+ (gbc t)\
+ (when (fboundp (quote si::sgc-on))
(si::sgc-on t))\
+ (setq compiler::*default-system-p* nil)\
+ " si::*system-directory* (quote (list ".lsp")))\
+ "$(OBJ)/$(SYS)/lib/sockio-c.o
$(OBJ)/$(SYS)/lib/cfuns-c.o $(OBJ)/$(SYS)/lib/libspad.a" \
+ nil))' | $(LISPSYS))
@ echo 6 ${SAVESYS} created
@ cp ${SAVESYS} ${AXIOMSYS}
@ echo 6a ${AXIOMSYS} created
*******************************************************
Here is the compiler::link version of the interpsys creation, which is
identical in structure to the depsys creation commented above.
*******************************************************
--- ./src/interp/nlib.lisp.pamphlet.orig 2005-01-05 00:06:03.000000000
+0000
+++ ./src/interp/nlib.lisp.pamphlet 2005-02-14 18:55:00.000000000 +0000
@@ -295,7 +295,16 @@
(defun rpackfile (filespec)
(setq filespec (make-filename filespec))
(if (string= (pathname-type filespec) "NRLIB")
- (recompile-lib-file-if-necessary (concat (namestring filespec)
"/code.lsp"))
+ (let* ((base (pathname-name filespec))
+ (code (concatenate 'string (namestring filespec) "/code.lsp"))
+ (temp (concatenate 'string (namestring filespec) "/" base ".lsp"))
+ (o (make-pathname :type "o")))
+ (si::system (format nil "cp ~S ~S" code temp))
+ (let ((compiler::*default-system-p* t))
(recompile-lib-file-if-necessary temp))
+ (si::system (format nil "mv ~S ~S~%"
+ (namestring (merge-pathnames o temp))
+ (namestring (merge-pathnames o code)))))
+ ;(recompile-lib-file-if-necessary (concat (namestring filespec)
"/code.lsp"))
;; only pack non libraries to avoid lucid file handling problems
(let* ((rstream (rdefiostream (list (cons 'file filespec) (cons 'mode
'input))))
(nstream nil)
*******************************************************
This is just a redirection of each code.lisp to a unique name, which
is needed when using compiler::link, as ld will insist that all
function names be unique, and GCL compiles init functions for each .o
file of the form init_filename. We're working on a better way to do
this in 2.7.0.
*******************************************************
Hope this is clarifying. Again, these blocks are only necessary at
present on ia64 mips alpha and hppa, though can be used elsewhere. We
intend this to be a temporary situation. The conventional
si::save-system dump is preferrable in many respects where available.
On 2.7.0, we have a feature :native-reloc which indicates when this
workaround is required.I can discuss development ideas we have in this
direction if interested.
Take care,
=============================================================================
> Other than these two blocks of code I've merged all of the other BSD
> changes back into my local tree.
>
>
> ======================================================================
> ======================================================================
> ======================================================================
>
> @ (cd ${OBJ}/${SYS}/bin ; \
> echo '(progn \
> (setq si::*collect-binary-modules* t) \
> (load "${OUT}/makedep.lisp") \
> (compiler::link \
> (remove-duplicates si::*binary-modules* :test (quote
> equal)) \
> "$(DEPSYS)" \
> (format nil "\
> (setq si::*collect-binary-modules* t) \
> (let ((si::*load-path* (cons ~S
> si::*load-path*))\
> (si::*load-types* ~S))\
> (compiler::emit-fn t))\
> (load \"$(OUT)/makedep.lisp\")\
> (gbc t)\
> (when si::*binary-modules* \
> (error si::*binary-modules*))\
> (setq si::collect-binary-modules* nil
> si::*binary-modules* nil)\
> (gbc t)\
> (when (fboundp (quote si::sgc-on)) (si::sgc-on
> t))\
> (setq compiler::*default-system-p* t)\
> " si::*system-directory* (quote (list ".lsp")))\
> "" \
> nil))' | $(LISPSYS))
>
>
> ======================================================================
> ======================================================================
> ======================================================================
>
> echo '(progn \
> (setq si::*collect-binary-modules* t)\
> (setq x si::*system-directory*)\
> (load "${OUT}/makeint.lisp")\
> (setq si::*system-directory* x)\
> (unintern (quote x))\
> (compiler::link \
> (remove-duplicates si::*binary-modules* :test
> (quote equal))\
> "$(SAVESYS)" \
> (format nil "\
> (let ((si::*load-path* (cons ~S
> si::*load-path*))\
> (si::*load-types* ~S))\
> (compiler::emit-fn t))\
> (setq si::*collect-binary-modules* t)\
> (setq x si::*system-directory*)\
> (load \"$(OUT)/makeint.lisp\")\
> (setq si::*system-directory* x)\
> (unintern (quote x))\
> (when si::*binary-modules* \
> (error si::*binary-modules*))\
> (setq si::collect-binary-modules* nil
> si::*binary-modules* nil)\
> (gbc t)\
> (when (fboundp (quote si::sgc-on))
> (si::sgc-on t))\
> (setq compiler::*default-system-p* t)\
> " si::*system-directory* (quote (list ".lsp")))\
> "$(OBJ)/$(SYS)/lib/sockio-c.o
> $(OBJ)/$(SYS)/lib/cfuns-c.o $(OBJ)/$(SYS)/lib/libspad.a" \
> nil))' | $(LISPSYS))
>
>
> Tim
>
>
>
--
Camm Maguire address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah