[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: float to integer conversion anomalies
From: |
Rob Mahurin |
Subject: |
Re: float to integer conversion anomalies |
Date: |
Tue, 30 Jun 2009 09:46:00 -0400 |
On Jun 30, 2009, at 5:16 AM, Levente Torok wrote:
Hi All,
May be I am commenting something that has already been reported or
even solved.
On octave-3.0.5 I found a somewhat inconsistent behavior.
octave:2> for i=.1:.1:2
i*10
endfor
ans = 1
ans = 2
ans = 3.00000000000000 <-????
This is because 0.3 does not have an exact floating-point
representation:
octave:2> sprintf("%.19f",.3)
ans = 0.2999999999999999889
Which is not a problem as long as I don't want to index with this.
But if I do I got error message.
octave:3> for i=.1:.1:2
z(i*10)=1;
endfor
error: expecting integer index, found 3.000000
error: assignment failed, or no method for `matrix = scalar'
error: evaluating assignment expression near line 4, column 8
error: evaluating for command near line 3, column 1
Try z(round(i*10)).
Rob
--
Rob Mahurin
University of Manitoba, Dept. of Physics & Astronomy
at: Oak Ridge National Laboratory 865 207 2594
Oak Ridge, Tennessee address@hidden