[Octave-bug-tracker] [bug #43728] rounding error for integer range multiplied with floating point

[Mike Miller]

Follow-up Comment #2, bug #43728 (project octave):

I think the difference seen here may be because Octave allows a range to be
multiplied by a scalar and the result is another range object. Multiplying the
integer ranges by 0.1 as you have is effectively the same as a floating point
range, with the drawbacks of which you were probably trying to avoid.

Specifically,


>> (-1:3) * 0.1 == -.1:.1:(3*.1)
ans =

   1   1   1   1   1

>> (-1:3) * 0.1 == [-.1, 0, .1, .2, .3]
ans =

   1   1   1   0   0

>> (-1:3) * 0.1 == [-1:3] * 0.1
ans =

   1   1   1   0   1

>> [-1:3] * 0.1 == [-.1, 0, .1, .2, .3]
ans =

   1   1   1   1   0



I would like to say this is working as expected, and what you really meant to
do was [-1:3]*0.1 to expand the range into a full matrix of integers first,
and then multiply elementwise by the scalar. Could be better documented
somehow?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?43728>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/