[Octave-bug-tracker] [bug #54562] numel should warn if the resulting double might be inaccurate

[Colin Macdonald]

URL:
  <http://savannah.gnu.org/bugs/?54562>

                 Summary: numel should warn if the resulting double might be
inaccurate
                 Project: GNU Octave
            Submitted by: cbm
            Submitted on: Thu 23 Aug 2018 05:54:32 PM UTC
                Category: Interpreter
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Inaccurate Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: GNU/Linux

    _______________________________________________________

Details:

  * With 64-bit indexing, the maximum index value (on my system) is
9.2234e+18

  * flintmax is 9.0072e+15.

  * numel returns type double.

Thus if a matrix is large (maybe sparse), it can easily have enough elements
such that numel is larger than flintmax and thus the numel value is not
precise.


>> [a, maxint]= computer ()
a = x86_64-pc-linux-gnu
maxint = 9.2234e+18
>> flintmax
ans = 9.0072e+15

>> A = sprand(1e10+123, 1e7+123, 1e-13);
>> j = int64(1e10+123) * int64(1e7+123)
j = 100001231230015129

>> n = numel(A)   # note, below maxint but above flintmax
n = 1e+17
>> i = int64(n)
i = 100001231230015136

>> i - j   # uh oh
ans = 7


Note: Matlab's numel gives a warning: "Number of elements exceeds maximum
flint 2^53-1".  Its still imprecise but at least user is informed.





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54562>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/