[Octave-bug-tracker] [bug #54100] fread using SKIP larger than zero is extremely slow

[Dan Sebald]

Follow-up Comment #14, bug #54100 (project octave):

OK, significant improvement.  However, I see your point.  Actually, looking at
this again, I bet the CPU consumer is not the read/skip, but the high
recurrence of calling "new".  When there is skip bytes, those small hunks are
all allocated dynamically.  I'm pretty certain that is a significant factor
because the OS has to do a lot of overhead finding and keeping track of the
memory.

Thinking about this, the current setup is optimum for when SKIP is very much
larger than block size.  But if SKIP is on the order of block size, it might
not be all that difficult to read in large quantities.  Mainly it would
involve doing a little extra math in computing the byte-size to read, and then
tossing out the skipped bytes right before 


            input_buf_list.push_back (input_buf);


which is quite easy because the buffer can be condensed in place with no new
memory.  I'll have a look sometime this week.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54100>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/