Re: [Axiom-developer] Re: Test summary

[Stephen Wilson]

On Wed, Dec 08, 2004 at 12:00:10AM -0500, root wrote: 

> (The "Daly Bug" message was so named because I needed a unique
> string and since I was the only one who would ever see the test
> results I used my name.  Apparently once IBM sold it NAG never
> changed it).

I enjoy hearing about these historical anecdotes. Like most large,
long-lived pieces of software, there is an interesting `culture'
associated with it. I remember Stephen Watt mentioning that when he
reads the Axiom source, he can see the changes in programming
methodology/practice which were common at the time the code was
written.


> If you look at the src/input/bugs.input.pamphlet file you'll see
> that I used to keep track of reported bugs, with failing examples,
> as well as fixes. These were kept around to ensure that they continued
> to work and that reported bug test cases were not lost.

Thank you for bringing this file to my attention. I had yet to read
it. There are a good number of examples which I can now keep in mind
as I continue my trek trough the compiler/interpreter code.


> So, the answer to your question is that, yes, we can do that and it
> should happen in the near future but in a more generally useful way.
> I will add both failure lists (unix and windows) to the README file
> on savannah at the next push.

I recall the regression tests, and their future reintegration, being
brought up in past postings. At least having the `Daly bug' list in a
canonical place until that time will give us some sort of baseline.

> In the long term I want axiom to run all the standardized tests in
> CATS. The CATS branch was on the previous arch server but I have yet
> to recreate that branch here. If there is any interest in developing
> better testing I'll do it now.
> 
> The challenge, of course, is to write the mathematics behind the
> tests into pamphlet files so the tests are properly documented
> and can be used with some confidence that the answer is correct.


Again, I recall earlier postings on this issue, and a comparison with
the NIST baseline for numerical code. I also recall mention of a grant
proposal w.r.t CATS. I would be dismayed if such a proposal could not
find sponsorship, considering its general usefulness and `essential'
nature.

I wish I had the mathematical sophistication to contribute to such an
effort. The only domain I have even modest experience in is with
respect to generic polynomial factorization, gcds, etc. However,
others have much more insight into `choosing the right polynomials'
which trigger exponential (say) branches in algorithms, or defeat some
common heuristic techniques. However, I do have some Aldor code which
implements functionality, and some recent algorithms, not present in
Axiom. When I get the chance to port it over perhaps I could
contribute to CATS in this area.


Sincerely,
Steve