[Octave-bug-tracker] [bug #63848] Mean: some single class inputs can hit precision limit with sum

[Nicholas Jankowski]

Follow-up Comment #10, bug #63848 (project octave):

yes, that matches with what I had to do to work around the int64/uint64
precision issues that couldn't just be cast to double for bug #54567.  

the consensus over on 61143 seemed to be to leave naïve summation as is
rather than implement a pairwise or similar algorithm, although it's been a
few years since that discussion. there's a linked matlab blog article that
indicates they implemented a more accurate block based summation across all
data types in 2020 after doing so for single in 2017, although as expected
they guard their blocksize determination routine.  I'm wondering if that
changes in any way the thoughts about a better summation algorithm in octave
(with the notion that octave may produce inaccurate answers for certain
summations/means/etc where matlab does not)


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63848>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/