[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] New module to perform decimal floating point arithmetic for
From: |
Ben Pfaff |
Subject: |
Re: [PATCH] New module to perform decimal floating point arithmetic for charts. |
Date: |
Tue, 13 Jan 2015 21:24:37 -0800 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Tue, Jan 13, 2015 at 07:03:17AM +0100, John Darrington wrote:
> This change adds a small module to perform floating point arithmetic
> with a decimal base, and uses it to calculate the graticule marks for
> charts.
>
> Rationale: Graticule marks want to be displayed in decimal. However
> not all decimal values can be represented in a double precision
> floating point binary. This approach pushes the precision loss from
> the value of the mark, to the position on the chart - we don't
> particularly care if a tick mark is a fraction of a pixel out of place,
> but we do care if 4.0 is displayed as 3.9999999999
Can you say a few words about the assumptions? At first glance, some of
the functions seem to suffer from catastrophic loss of precision upon
integer overflow, e.g. doesn't that happen in decimal_multiply() if the
intermediate result is greater than ORD_MAX? (Is some kind of care
taken to make sure that this cannot happen?)