[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow
From: |
Dan Sebald |
Subject: |
[Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly |
Date: |
Tue, 31 Jul 2018 21:27:12 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0 |
Follow-up Comment #19, bug #54405 (project octave):
@JWE, just a thought. Here's what gcc compiler does when trying to take the
absolute value of minimum int64 value:
CXX libinterp/corefcn/libinterp_corefcn_libcorefcn_la-oct-stream.lo
/home/sebald/octave/octave/octave/libinterp/corefcn/oct-stream.cc:6610:55:
warning: integer constant is so large that it is unsigned
std::cerr << "labs(-9223372036854775808) = " << labs(-9223372036854775808) <<
"
Somehow it's figuring out from the ASCII string that it won't work. Octave
could do a similar check on a manually entered integer index value rather than
convert it to float. That is, say the user types:
zeros(92233720368547758083333)
The error could be that the index is too large, rather than out of memory.
Huh, I just tried this and
octave:1> zeros(92233720368547758083333)
ans = [](0x0)
octave:2> zeros(9223372036854775808333)
ans = [](0x0)
octave:3> zeros(922337203685477580833)
ans = [](0x0)
octave:4> zeros(92233720368547758083)
ans = [](0x0)
octave:5> zeros(9223372036854775808)
ans = [](0x0)
octave:6> zeros(922337203685477580)
error: out of memory or dimension too large for Octave's index type
octave:6> zeros(922337203685477587)
error: out of memory or dimension too large for Octave's index type
octave:6> zeros(922337203685477588)
error: out of memory or dimension too large for Octave's index type
octave:6>
Possible bug?
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?54405>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, (continued)
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dmitri A. Sergatskov, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly,
Dan Sebald <=
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Rik, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31
- [Octave-bug-tracker] [bug #54405] octave_idx_type index integer overflow math check doesn't work correctly, Dan Sebald, 2018/07/31