[Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data

[Arun Giridhar]

Follow-up Comment #6, bug #63803 (project octave):

I see what you mean. Changing that line to read:

const bool *mtmp = m.data ();

eliminates one copy. For the same 5 GB array:

octave:4> save -binary test3.mat
../libinterp/octave-value/ov-bool-mat.cc:369 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:374 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:379 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:384 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:389 Usage: 10571063296
../libinterp/octave-value/ov-bool-mat.cc:395 Usage: 10571063296
../libinterp/octave-value/ov-bool-mat.cc:400 Usage: 10571063296


Further eliminating htmp and writing the file directly as a reinterpret_cast
from mtmp removes all temporaries:

octave:1> arr = false (1e5, 5e4);
octave:2> whos
Variables visible from the current scope:

variables in scope: top scope

  Attr   Name        Size                     Bytes  Class
  ====   ====        ====                     =====  ===== 
         arr    100000x50000             5000000000  logical

Total is 5000000000 elements using 5000000000 bytes

octave:3> nnz (arr)
ans = 0
octave:4> save -binary test4.mat
../libinterp/octave-value/ov-bool-mat.cc:369 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:374 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:379 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:384 Usage: 5571059712
../libinterp/octave-value/ov-bool-mat.cc:389 Usage: 5571059712


The files test3.mat and test4.mat are identical, so we don't need the htmp
intermediate to write booleans as characters.

$ diff test3.mat test4.mat 
$


Here's a fix for this function, which eliminates all temporaries:

diff -r 3e4e74ad8fd7 libinterp/octave-value/ov-bool-mat.cc
--- a/libinterp/octave-value/ov-bool-mat.cc     Mon Feb 13 11:10:00 2023
-0800
+++ b/libinterp/octave-value/ov-bool-mat.cc     Tue Feb 14 13:57:31 2023
-0500
@@ -362,14 +362,9 @@ octave_bool_matrix::save_binary (std::os
     }
 
   boolNDArray m = bool_array_value ();
-  bool *mtmp = m.fortran_vec ();
+  const bool *mtmp = m.data ();
   octave_idx_type nel = m.numel ();
-  OCTAVE_LOCAL_BUFFER (char, htmp, nel);
-
-  for (octave_idx_type i = 0; i < nel; i++)
-    htmp[i] = (mtmp[i] ? 1 : 0);
-
-  os.write (htmp, nel);
+  os.write (reinterpret_cast<const char*> (mtmp), nel);
 
   return true;
 }




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63803>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/