[Octave-bug-tracker] [bug #63791] movfun - high intermediate memory use for moderate data input resulting in out of memory errors

[Nicholas Jankowski]

Update of bug #63791 (project octave):

                Severity:              3 - Normal => 2 - Minor              
                Priority:              5 - Normal => 3 - Low                
                  Status:               Need Info => Confirmed              
        Operating System:       Microsoft Windows => Any                    
                 Summary: movmean - out of memory or dimension too large for
index type => movfun - high intermediate memory use for moderate data input
resulting in out of memory errors

    _______________________________________________________

Follow-up Comment #19:

not sure the bug needs to be closed. perhaps we can rename it to indicate that
there is an internal memory multiplication issue that may be worth looking
into eventually. Octave is vectorizing the solution, which multiplies the
array size internally before collapsing it again for the final solution. This
is fast, especially in Octave which performs very poorly with looped code
compared to matlab. But it comes at a windowsize x inputsize memory cost. 
There have been a few cases in the past where code choices have created
unnecessary copies of data in memory that were later fixed. Perhaps one can be
found for movefun. 

Looking here, slcidx is a sliding continuous index, stored as a double with
max value = numel(x).  To accommodate the 64bit int index limit i guess that
can't be reduced just by choosing a smaller int type. But maybe there are
better ways to represent the window, perhaps just by the endpoint values
depending how it is later used.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63791>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/