[Axiom-developer] Silver

[daly]

SILVER

The latest, merged version will momentarily be available as
a silver version. 

It contains my attempt to merge the various versions. It is a couple
months behind "the leading edge" so not everything got picked up. It
does not contain the new GNU autoconf mechanism yet because (a) I don't
understand it and (b) it isn't documented to a level that a new
developer could begin to change it.

I picked up all of the changes I understood and left others alone.



REGRESSION TESTING

Silver contains 2 new tools for regression testing the system. 

First, I've written a regression program for the input files so we can
easily spot input file failures automatically (the results are stored
in "regress" files in the int/input directory). I've rewritten many of
the src/input files into the new format. Basically these files store
the expected result of a computation in a specially constructed comment.
The code and its associated documentation is in src/interp/regress.lisp
The new input files contains lines that read:

--S n of m                       <== this starts a test
(some axiom command)             <== the command to test
--R                              <== the expected results
--R the axiom output
--E n                            <== the end of the test

As each input file is run the actual results are compared with
the expected results and the pass/fail information is written
to a regress file. So

foo.input.pamphlet -> foo.input -> foo.output -> foo.regress

Also, removing the int/input subdir and rerunning make will now
rerun all of the input files.



Second, I've written a global regression check that uses md5 hashing
to find changed files. This is automated in the src/regress/REGRESS
script. This script rebuilds axiom and then computes the md5 hash
for each file and compares it to the stored md5 hash. Thus any change
in a file will generate a different hash code. This allows us to see
what effect each change has on every file in the system. So

src/regress/REGRESS is a shell script to clean the system, remake
the system, and then compute the md5 hash into allnew.md5 with one
hash line per file. Finally we compre allnew.md5 with src/regress/all.md5
and any differences mean that the file was changed. This simple system
is useful to see what a change potentially breaks.

It also sends mail when the build completes so you might want to
change the last line to send the mail to your own userid. This
is useful if you kick off a build and go to work.



ALGEBRA

Silver does not contain all of the algebra changes. I have been pondering
ways to ensure that the algebra does not break. We don't want to fix
one thing and break others so I'm taking a very conservative approach.

I think we might need an algebra-change review board of some sort.
I know I'm not competent to judge all of the algebra changes. 

I also believe that changing an algebra file should be accompanied by
a larger effort at documenting that particular file. After all, if
we've spent the time to isolate a problem we can profit by writing
down the things that we know about the algebra despite the time and
overhead it might take.  We also need associated unit-tests for each
function and regression tests for expected output.

One new algebra file was documented (quat.spad) and more work will
go into the documentation. I hope to have the spad input lines
automatically executed as part of an automated test suite generated
directly from the pamphlet files. This is not yet published but is
a work in progress.




LOCATION

The silver version is in two places, SVN and git, and I'll be continuing 
development against both places in parallel. It is not yet ready to 
become Gold.

THE SVN VERSION

I put the silver version into daly/axiom on sourceforge. It will
be changed regularly from now on. Please look at it and propose
changes (with diff-Naur). 

THE GIT VERSION

I've put up a git repository of the current "silver" version
of Axiom. In order to use this you need either an ssh-key or
you can use git's password (linus). If you already have an ssh-key
register on axiom-developer.org you should be able to clone.
If you don't want to use the password you can send me an ssh-key. Do:
   ssh-key -t dsa
and send me the generated file (your .ssh/id_dsa.pub)

Everyone has read/write access to this repository.


WHY?

If you watched the video link I sent you know that git is a fully
distributed development system, unlike arch, cvs, or svn. Thus
every copy is a "root" copy and no copy is special, except by
convention. This differs considerably from the "star"-network,
master-repository view. Thus you all have THE silver version.

See <http://git.or.cz>

The seed repository is on axiom-developer.org under the user 'git'.
You can get your own repository with:

git-clone ssh://address@hidden/home/git/silver silver

Once you do this you can make modifications to this tree.




YEAH, BUT WHY?

Well, git is fast. And git allows you to do work quite easily in
multiple branches simultaneously. So each branch can be a separate
idea.  Branches are dirt-cheap (a few bytes). And git is blindingly
fast. And git does not store a duplicate copy, nor does it copy files
so it uses less than half the space. Did I mention git is unbelievable
fast? etc., etc., etc... all the usual religious noise.




YEAH, BUT SERIOUSLY, WHY?

Because git eliminates me from being "special" in the world
of development. CVS and SVN put the repository in the center 
of the world and everyone has to play. With git the whole idea
of a "silver" version becomes a non-issue. So I'm no longer
the single point of failure. In a fully distributed system there
is no single point of failure.

Now all I need to do to get a changeset is to "pull" from your
repository and the merge is done automatically. Now you have
the master copy and I just bring my copy up to date. Sweet, no?

Git will change the way you work to something much better.




HOW?

1) get the code
  git-clone ssh://address@hidden/home/git/silver silver

  the clone is a full, independent git repository. people can clone
  your whole repository, pull changes from your repository, or push
  changes to your repository. thus you are the center of the world.

2) branch the code
  git-branch mybranch

  if you work on a single problem (eg an algebra bug) in a single
  branch you will naturally accumulate a changeset. since git will
  not copy files the branch is only a few bytes big. you can have
  as many branches as you wish so each branch can be a single idea
  or problem.

3) list the branches
  git-branch

  you can see all of the branches you have created

4) change to the new branch
  git-checkout mybranch

  if mybranch is related to one single change then you can work
  on that change locally independent of all other work.

5) change something
  echo "it works" >>FAQ

  here we modify the FAQ file

6) remember the change
  git-add test

  NOTE: git does not know about files, it knows about changes.
  If you change something you need to tell git you want to add that
  change to the set of changes to keep. So "git-add" means ADD THE
  CHANGE, NOT ADD THE FILE. git only cares about content. it does
  not track files. This is different from other source code control.

7) commit the change on this branch
  git-commit -a -m"mybranch test"

  All of the changes we have made (and have selected with git-add)
  will be collected into a commit. The -a says to take them all and
  the -m is an inline comment.

  Notice that commit does NOT use the network since you ARE
  the master copy and this is a local branch.

8) change to the master branch
  git-checkout master

  Now that we've made our glorious change in isolation it is time to
  merge it back onto the main tree. So first we switch to the master
  tree (or whatever tree we want)

8) merge the branch
  git-merge mybranch

  And we merge the master and mybranch trees

9) see what changed
  git-log

  We can see what changed. The commit log tag can tell you a lot of
  detail. Notice that there are lines that read something like:

  commit 1e6bb35c9089a197513decd4e988381205120260
  ...
  commit 731bed1e6bb35c9089a197513decd4e988381205
  ...

10) see the details of a change
  git-show 731bed

  This shows you the change 731bed1e6bb35c9089a197513decd4e988381205

  Notice that you can abbreviate the long md5 hash numbers as long as
  they are unique. The md5 hash algorithm is even and almost always unique
  so very short numbers suffice. You can also make up your own names
  (say v3.2) and use them instead. RTFM.

11) see the change in a range of changes
  git-diff 1e6bb3--731

  This will show you a diff -Naur difference within this range

12) commit the change
  git-commit -a

  We commit the changes locally so they will be remembered. 
  Notice that commit does NOT use the network since you ARE
  the master copy.

13) delete the branch
  git-branch -d mybranch

  We've solved the problem so clean up the pile.

14) push the change back to axiom-developer.org
  git-push

  If you wish you can reach out and change the repository on 
  axiom-developer.org with our changeset. You can also "push" to
  someone else's repository. Or they can "pull" from yours. So
  all you need to do is skip this step, post a message on 
  axiom-developer, and let me pull your changeset.

Tim