[Octave-bug-tracker] [bug #35679] median (rand (1, 1, 1, 3), 4) segmentation fault

[Rik]

Follow-up Comment #10, bug #35679 (project octave):

It's possible that there is more to the issue, but the trimming of singleton
dimensions was definitely being done in the wrong place.  When I single
stepped through the code what I discovered was that the variable stride was
getting set to an incorrect value.  Backtracking, I found that stride was
being set incorrectly in this loop.


octave_idx_type stride = 1;

for (int i = 0; i < dim; i++)
  stride *= dv(i);


The problem is not with this code itself, but with the fact that dim was based
on the original size of the dimension vector dv and not the current size,
which may be smaller thanks to chop_singleton_dimensions().  The for loop was
overstepping the actual bounds of dv and indexing into some unknown region of
memory which resulted in a crazy value for stride.

Separately, I too noticed that in the original code any odd value for the
number of elements produces a segfault while any even value is acceptable.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?35679>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/