[Octave-bug-tracker] [bug #61087] PLOT function produces incorrect graph.

[Trevor]

Follow-up Comment #2, bug #61087 (project octave):

Oh that's very disappointing, but thanks for clarifying Dimitri.

Is there a plan to fix the problem ?


I do wonder though if a trick is being missed here.  I agree that a single
precision library will manifest the problem, but it's not really the CAUSE of
the problem in my view.

When it comes down to it a good high res display hardware has perhaps
7680x4320 pixels, and that defines the plot resolution, not single or double
precision.  The job of the plotting subsystem is to map the graphical data to
that limited display range by normalising it (subtract the mean offset and
scaling).  Given the limited hardware resolution, single precision arithmetic
has sufficient resolution to correctly do that.

However the PLOT function takes in double precision data and passes it to
OpenGL, which apparently uses single precision.  If the data is not
appropriately normalised (subtract the mean offset and scale) at the double to
single conversion then detail will be lost at that stage, and that I suspect
is where the problem lays.  

I'd suggest that the PLOT function is not appropriately normalising the data
before it is converted to single precision.  The normalisation should only
need to ensure the detail fits into a single precision number, the
normalisation to the display hardware resolution should be handled in the
graphics library itself.  The actual value can still be shown by axis labels.

The issue is really that the PLOT function accepts double precision data, but
does not correctly map double precision data to the display, which should be
possible.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61087>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/