[Axiom-developer] Re: [sage-devel] Re: doctest failures due to rounding errors on Solaris.

[Tim Daly]

William Stein wrote:
> On Thu, Dec 31, 2009 at 9:13 PM, Tim Daly <address@hidden> wrote:
>   
> > Dr. David Kirkby wrote:
> >     
> > > rjf wrote:
> > >
> > >       
> > > > On Dec 31, 11:15 am, "Dr. David Kirkby" <address@hidden>
> > > > wrote:
> > > >
> > > >
> > > >         
> > > > > > RJF
> > > > > >
> > > > > >             
> > > > > The point you are missing is that we want to compare the output what Sage prints
> > > > > to a human.
> > > > >
> > > > >
> > > > >           
> > > > The point you are missing is that the following item, which presumably
> > > > could be printed by Sage,
> > > > is perfectly readable to a human:
> > > >
> > > > 6121026514868073 * 2^(-51).
> > > >
> > > > It exactly dictates the bits in an IEEE double-float, and does not
> > > > require any conversion from binary
> > > > to decimal. It does not need rounding.  This kind of representation
> > > > does not have any hidden unprinted digits.  It does not ever need to
> > > > be longer because of delicate edge conditions of certain numbers.
> > > >
> > > > It happens to evaluate to
> > > > APPROXIMATELY   2.718281828459045
> > > >
> > > >         
> > > Sure, Sage could print that. It would also be worth printing the sign bit, so we
> > > could verify the values of
> > >
> > > 1) Sign bit
> > > 2) Significand
> > > 3) Exponent.
> > >
> > > All of those could be correct. But there is still the software which does the
> > > non-trivial task of converting that into the base 10 representation used by
> > > humans. Then in additon to that, there is the software which takes a base 10
> > > number, shows it with the Sage prompt, adding carriage returns etc where
> > > necessary. All of these can go wrong.
> > >
> > > I would think in an almost ideal world, the test would be done at a higher
> > > level, using hardware/software which checked what the monitor actually
> > > displayed. That's not quite as easy to do though.
> > >
> > > Even better would be some way to scan the brain of the user to see what he/she
> > > believes Sage is showing. Perhaps we use a font that is not very good, so
> > > despite being displayed properly, it misunderstood.
> > >
> > > Given most of time people want to see a base 10 representation of a number, and
> > > not a base 2, base 16 or IEE 754 representation, I believe most testing should
> > > be done at the base 10 level.
> > >
> > > If there is a reason for testing the IEEE 754 representation as first choice,
> > > then you have yet to convince me of it.
> > >
> > >
> > > Dave
> > >
> > >
> > >
> > >       
> > Dave,
> >
> > Axiom has the same issues.
> >
> > My take on this is that what you check depends on the reason you are
> > checking.
> > If you are generating the output for human use (e.g. a table) then you
> > want decimal.
> > If you are generating the output for regression testing (e.g. checking
> > the answers on
> > multiple hardware) then you probably want Fateman's solution.
> >
> > Tim
> >     
>
> The output is used both for human use and for regression testing.  Its
> primary use is human -- it's an example in the Sage reference manual:
>
>    sage: float(e)
>    2.7182818284590451
>
> This is something a user will look at when reading the documentation
> for some function.  It illustrates what happens when they convert the
> symbolic constant e to float.
>
> William
>
>   
And therein lies the problem. We use a regression that does a comparison 
of the
printed representation of the output of the run with a stored copy of 
the output.

All of our regression tests were passing until I installed another, 
unrelated program.
Suddenly about 30 regression tests started failing. It turns out that 
the unrelated
program upgraded one of the system libraries. The net effect of that 
change was
to cause the last digit in the output to "wobble" so that some of the 
table values
differ in the nth place (20th, 30th, or thereabouts digit). This caused 
the regression
comparisons to fail.

Common lisp will give you the exact bit pattern of the float and this value
does not wobble so the text comparison succeeds with both the old and
the new libraries against the bit pattern.

So I can tell you from experience that what you would like to do is not
going to succeed.

Our solution to the human vs regression problem is to include the stable
bit values in the actual compare and keep the human values in a latex
table. This is easy to do with literate input.

Tim