|
From: | David Ayers |
Subject: | Re: RFA/PATCH gtest - NSNumber float != double |
Date: | Wed, 22 Jan 2003 10:09:58 +0100 |
User-agent: | Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3a) Gecko/20021212 |
Alexander Malmberg wrote:
David Ayers wrote: [snip]Hmm, I'll take your word for it. I just recalled back in the dark ages someone told me (or I read it somewhere) that 25.1 would somehow be represented as 2.51 * 10^1 and that a float or double only stored the mantissa and exponent which shouldn't result in any discrepancies here.It does, but in base 2, and the fraction 25.1/16 can't be represented exactly in base 2; you get 2^4 * 1.10010 0011 0011 0011 ...
Thanks,and verified with OPENSTEP 4.2. The "patch" is withdrawn. I apparently forgot the part about base for floats and doubles also being 2. But I guess it is probably the only way to handle these numbers somehow efficiently for (floating point) processors.
Thanks for the lesson :-) Cheers, Dave
[Prev in Thread] | Current Thread | [Next in Thread] |