Re: [Axiom-developer] GSoC CAS-neutral test suite project

[Tim Daly]

Yes.


A trigonometry test suite would be interesting. I'm a member of
the Numerical Mathematics Consortium
(http://www.nmconsortium.org/index.aspx)
and they recently published a technical specification for trig functions.
(http://math.mit.edu/~numericalmath/nmc/index.php/Technical_Specification)

If you look at Appendix A you can see the table of branch-cut choices.
Since this consortium also includes companies such as Maple it would be
a good place to devote some "system independent" effort. It is on my "todo"
list to make a test suite based on this standard so I'd be happy to help
guide a student, even if someone else is the public lead.

In particular, just developing a test suite that would highlight the
branch cut choices made by a variety of systems would be very valuable.
I know that MMA and Maxima tend to agree and Axiom and Maple tend to
agree but we both make different choices.

In addition, we could poll the NMC for students. There are people in
the consortium with wide-ranging contacts, like Inria, Maplesoft,
National Instruments, etc.




Alternatively I think a test suite of sequences and series might be useful.
Or contact William Kahan about numerical library standards (e.g. BLAS? 
ATLAS?)
that might need standards. Richard Fateman is also a good contact as he is
one of the Maxima authors and well versed in things like floating point 
formats.
Stephen Watt is also another source of potential standards. James Davenport
is a lead on the OpenMath standards committee and could suggest potential
cross-platform standards.




Ted Kosan wrote:
> On Wed, 03 Mar 2010 06:40:49 -0800, Tim Daly
> wrote to [Axiom-developer] Re: [sage-devel] Randomised testing against
> Mathematica
>
>   
> > There are two test suites with validated results at
> > http://axiom-developer.org/axiom-website/CATS/
> >
> > The CATS (Computer Algebra Test Suite) effort targets
> > the development of known-good answers that get run
> > against several systems. These "end result" suites test
> > large portions of the system. As they are tested against
> > published results they can be used by all systems.
> >
> > The integration suite found several bugs in the published
> > results which are noted in the suite. It also found a bug
> > introduced by an improper patch to Axiom.
> >
> > It would be generally useful if Sage developed known-good
> > test suites in other areas, say infinite sequences and series.
> > Perhaps such a suite would make a good GSOC effort with
> > several moderators from different systems.
> >
> > I have done some more work toward a trigonometric test
> > suite. So far I have found that Mathematica and Maxima
> > tend to agree on branch cuts and Axiom and Maple tend
> > to agree on branch cuts. The choice is arbitrary but
> > it affects answers. I am having an internal debate about
> > whether to choose MMA/Maxima compatible answers just to
> > "regularize" the expected results users will see.
> >
> > Standardized test suites give our users confidence that
> > we are generating known-good results for some (small)
> > range of expected inputs.
> >
> > An academic-based effort (which Axiom is not) could
> > approach NIST for funding an effort to develop such
> > suites. NIST has a website (http://dlmf.nist.gov/)
> > Digital Library of Mathematical Functions. I proposed
> > developing Computer Algebra test suites for their
> > website but NIST does not fund independent open source
> > projects. Sage, however, could probably get continuous
> > funding to develop such suites which would benefit all
> > of the existing CAS efforts.
> >
> > NSF might also be convinced since such test suites raise
> > the level of expected quality of answers without directly
> > competing against commercial efforts. I'd like to see a
> > CAS testing research lab that published standardized
> > answers to a lot of things we all end up debating, such
> > as branch cuts, sqrt-of-squares, foo^0, etc.
> >
> > Tim Daly
> >     
>
> MathPiper (http://mathpiper.org) is the CAS which is used by GeoGebra
> (http://geogebra.org) and GeoGebra was recently accepted as a Google
> Summer of Code project.  The GeoGebra project has invited the
> MathPiper project (which I lead) to participate in their GSoC effort
> and I immediately thought that your idea of a CAS-neutral test suite
> would be a good project candidate.
>
> Would you be interested in helping to locate and mentor a GSoC student
> to work on a CAS-neutral test suite which is based on the ideas you
> discussed in the above email?
>
> Ted Kosan
>
>
> _______________________________________________
> Axiom-developer mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/axiom-developer
>
>