[Octave-bug-tracker] [bug #44206] test complex.tst fails in MXE-octave

[John W. Eaton]

Follow-up Comment #4, bug #44206 (project octave):

In the sort function, Octave is just using the simple comparison operator that
is defined in oct-sort.cc.  That doesn't account for NaNs or the arg of a
complex value.  But the issorted function is using the nan_ascending_compare
function that is defined in Array-C.cc:


static bool
nan_ascending_compare (const Complex& x, const Complex& y)
{
  bool retval = false;

  if (xisnan (y))
    retval = (! xisnan (x));
  else
    {
      double xabs = std::abs (x);
      double yabs = std::abs (y);

      if (xabs < yabs)
        retval = true;
      else if (xabs == yabs)
        {
          double xarg = arg (x);
          double yarg = arg (y);

          xarg = (xarg == -M_PI) ? M_PI : xarg;
          yarg = (yarg == -M_PI) ? M_PI : yarg;

          retval = xarg < yarg;
        }
      else
        retval = false;
    }

  return retval;
}


The comparison that fails is


(-1,-0) < (-1,0)


It should return false, but returns true on Windows systems, apparently
because the xarg and yarg values that are computed are stored in registers
that end up not being exactly equal to M_PI or -M_PI.  It's hard to observe
because the values are optimized out so I can't look at them with gdb and
attempting to print them causes the compiler to store them, which makes the
function "work".

I'm checking now to see whether declaring xarg and yarg as volatile will fix
the problem.  I assume it will, but this doesn't seem like a great fix in any
case because we are still relying on exact comparison of a value returned from
a function with a compile-time constant.  Hmm.

Another question is why the sort function is not using the special
nan_ascending_compare function.  Shouldn't it be using that one so that we get
complex comparisons right?  And, in any case, shouldn't the comparison
function that is used by sort be consistent with the one used by issorted?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?44206>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/