[Octave-bug-tracker] [bug #42579] textscan error out of memory or dimension too large

[Chris Mumby]

Follow-up Comment #4, bug #42579 (project octave):

Thanks for the reply.

I confirm that if I reduce the amount of lines that it starts to work, I
scaled the 37mb file back to 30mb and it worked. However, you are right, the
memory usage in Windows for octave-cli-3.8.1.exe jumps to about 1.3GB and then
goes down, and then starts heading towards 1GB; therefore, the textscan
function is not memory efficient (does seem strange given that you tell
textscan the format). I suspect that if there was a 64bit version for windows
the 37mb might work as there would be more than 2GB of process memory
available to Octave.

It's not a bug per se, but 36mb ascii file is nothing serious, Matlab has not
problem, and in other languages it isn't a problem (37mb is tiny), so I do see
it as a problem. If textscan can't be fixed in its current form, and a C++
rewrite isn't an option, this textscan limitation should be documented, or the
recommendation not to use textscan. I am trying to have .m files that can be
used with Matlab (what we use now) and Octave, while I have had some success,
in general, it is proving difficult and it is a barrier (so I can't use
csv2cell). For large datasets I do use something more efficient, like binary
file formats, but 37mb in text does not need to do this conversion. 

I do hope textscan can be fixed. I have programmed quite a bit on Windows, but
not on Linux, (and I can't find instructions on how to compile on Windows) so
if there is more information needed I might be able to help.



    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?42579>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/