[Octave-bug-tracker] [bug #63848] Mean: wrong result for large single vectors

[Dmitri A. Sergatskov]

Follow-up Comment #2, bug #63848 (project octave):

A quick-n-dirty fix is toadd "double" qualifier to all 
calls to sum() in the mean.m file:

+verbose+
diff -u /usr/local/share/octave/8.0.91/m/statistics/mean.m 
../scripts/statistics/mean.m 
--- /usr/local/share/octave/8.0.91/m/statistics/mean.m  2023-02-27
13:04:16.784440819 -0500
+++ ../scripts/statistics/mean.m        2023-02-28 10:45:49.227600518 -0500
@@ -136,7 +136,7 @@
       n = numel (x(:));
       if (omitnan)
         idx = isnan (x);
-        n -= sum (idx(:));
+        n -= sum (idx(:),'double');
         x(idx) = 0;
       endif
       y = sum (x(:), 1) ./ n;
@@ -147,10 +147,10 @@
       n = size (x, dim);
       if (omitnan)
         idx = isnan (x);
-        n = sum (! idx, dim);
+        n = sum (! idx, dim,'double');
         x(idx) = 0;
       endif
-      y = sum (x, dim) ./ n;
+      y = sum (x, dim,'double') ./ n;
     endif
 
   else
@@ -166,10 +166,10 @@
       n = size (x, dim);
       if (omitnan)
         idx = isnan (x);
-        n = sum (! idx, dim);
+        n = sum (! idx, dim,'double');
         x(idx) = 0;
       endif
-      y = sum (x, dim) ./ n;
+      y = sum (x, dim,'double') ./ n;
 
     else
 
@@ -186,10 +186,10 @@
           n = numel (x(:));
           if (omitnan)
             idx = isnan (x);
-            n -= sum (idx(:));
+            n -= sum (idx(:),'double');
             x(idx) = 0;
           endif
-          y = sum (x(:), 1) ./ n;
+          y = sum (x(:), 1,'double') ./ n;
 
         ## for 1 dimension left, return column vector
         case 1
@@ -200,7 +200,7 @@
             if (omitnan)
               x_vec = x_vec(! isnan (x_vec));
             endif
-            y(i) = sum (x_vec, 1) ./ numel (x_vec);
+            y(i) = sum (x_vec, 1,'double') ./ numel (x_vec);
           endfor
           y = ipermute (y, [misdim, dim]);
 
-verbose-

Be aware that this will make mean() to output "double" even for "single"
input.

Dmitri.
-- 



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63848>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/