[Octave-bug-tracker] [bug #45945] Missing Matlab's special support for integer literals

[Amro]

Follow-up Comment #1, bug #45945 (project octave):

This really comes down to a weakness in the MATLAB/Octave language syntax;
there is no (documented) way to directly specify integer literals. Other
languages have suffixes like U, L, and LL for this purpose...

I guess MATLAB solved this issue by having a special case when calling the
u?int(8|16|32|64) functions specifically with a literal integer.


% MATLAB
>> format hex
>> intmax('uint64')
ans =
   ffffffffffffffff
>> uint64(18446744073709551614)    # special case in MATLAB
ans =
   fffffffffffffffe
>> uint64(18446744073709551614.0)  # usual interpretation
ans =
   ffffffffffffffff


Octave however isn't doing anything special, and instead first interprets the
literal number as double type (as usual), and then casts the temporary value
to uint64. There is an entry on the wiki FAQ:
http://wiki.octave.org/FAQ#How_do_I_make_Octave_use_more_precision.3F

Now the reason the number comes out as ...615 instead of ...614 is obviously
explained by the limitations of IEEE754 double-precision floating-point
format. Integers up to 2^53 can be exactly represented as double, above that
there is gap of one between every representable integer-valued floating-point.
The gap doubles every time we go up by one exponent (2^54, 2^55, etc...). In
this case the number you used is one less than intmax('uint64') (2^64-2 to be
exact). So according to the default "round-to-nearest" floating-point rounding
mode, it ends up getting rounded along with a bunch of nearby numbers all
rounded to the same value of ...615 (I think at that range the gaps are 2048
and 4096, half of 2048 rounded up, and half of 4096 rounded down, middle value
resolved with a tie-breaker rule). 


% MATLAB
>> format hex
>> [18446744073709550591; 18446744073709550592; (18446744073709551614);
18446744073709553664; 18446744073709553665]
ans =
   43efffffffffffff
   43f0000000000000
   43f0000000000000   % your value
   43f0000000000000
   43f0000000000001


As for STR2NUM, it actually uses EVAL to parse the input (both MATLAB and
Octave), so it's expected to behave similar to typing the commands on the
prompt directly..

The solution in this case is to use a bignum library (like the Symbolic Math
Toolbox in MATLAB). I think there are some wrappers for GNU GMP and GNU MPFR
libraries for Octave that provide arbitrary precision integer and float
numbers.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?45945>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/