|
From: | Rik |
Subject: | [Octave-bug-tracker] [bug #58147] Matlab 2020 compatibility: dec2bin, dec2hex shouldn't error on negative inputs |
Date: | Wed, 22 Apr 2020 20:40:38 -0400 (EDT) |
User-agent: | Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko |
Follow-up Comment #12, bug #58147 (project octave): The current Octave implementation has a problem with numbers greather than flintmax(). We should either restrict the range of inputs and issue an error if the input is outside that range, or modify the algorithm. A test case is dec2bin (-2^31 -1) which should return "1111111111111111111111111111111101111111111111111111111111111111" , but instead returns "1111111111111111111111111111111110000000000000000000000000000000" The trouble is this line, I think d(lt_zero_idx) += double (intmax ("uint64")) + 1; intmax() for "uint64" is larger than flintmax() (2^53) for double values and so precision is lost. _______________________________________________________ Reply to this item at: <https://savannah.gnu.org/bugs/?58147> _______________________________________________ Message sent via Savannah https://savannah.gnu.org/
[Prev in Thread] | Current Thread | [Next in Thread] |