[Octave-bug-tracker] [bug #63848] Mean: wrong result for large single vectors

[Rik]

Follow-up Comment #4, bug #63848 (project octave):

Matlab seems to be following a strategy where they

1. Record class of input
2. Convert input to double
3. Perform calculation
4. Convert result back to input class

We could do that in Octave as well.  Right now we rely on the underlying
functions like sum() which automatically perform operations in the class of
the input for floating point values.

In this particular case, it would advantageous not to create a duplicate of
the original matrix, but merely of type double, which would use a lot of
additional memory.  Instead, as Dmitri suggested, add "double" the function
calls to sum() and then convert back to result class at the end of the
function with a call to cast().


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63848>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/