[Octave-bug-tracker] [bug #62283] max (a, b) produces unintuitive results when mixing integers and floating point values

[Rik]

Update of bug #62283 (project octave):

                Category:           Documentation => Octave Function        
              Item Group: Unexpected Error or Warning => Documentation        
 
                  Status:                    None => Confirmed              
                 Release:                   6.4.0 => dev                    
        Operating System:       Microsoft Windows => Any                    
                 Summary: max (a, b) gives neither a nor b but a random third
value => max (a, b) produces unintuitive results when mixing integers and
floating point values

    _______________________________________________________

Follow-up Comment #1:

Confirmed.  I re-titled the report to be a little clearer.  A agree that the
behavior appears strange.  It is, however, explainable and also
Matlab-compatible.

I can explain what is happening for this code


a = int8 (100);  b = 200;  max (a, b)


First, Octave/Matlab always returns a value of integer type when a mixed
computation of floating point and integer values is used. 
 For example,


y = int8 (2) + double (2)
y = 4
class (y)
int8


When Octave calculates


max (int8 (100), double (200))


it respects this rule and returns a result of class int8.

The first part of the calculation is to force the floating point number to be
of type int8 but since 200 is greater than the maximum representable value it
just returns that max value which is 127


int8 (200)
ans = 127


After that, the rest of the calculation is straightforward


max (int8 (100), int8 (127))
ans = 127





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?62283>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/