[Octave-bug-tracker] [bug #43327] dir function is too slow

[Rik]

Follow-up Comment #9, bug #43327 (project octave):

This bug is very old, but there have been developments since 2014.  In
particular, see bug #50416 from 2017 where there was another report about slow
processing times for dir.  I recoded dir.m at that time to be ~4X faster.  I
checked in another changeset today that improves performance by 10%
(https://hg.savannah.gnu.org/hgweb/octave/rev/2c4759c8239c).

At this point, I don't believe there is much more that can be done at the
m-file level.  Using the profiler, 2/3rds of the running time is associated
with dir.m itself, which really means the for loop within dir.m. 

Profiler results for all functions > 1% of run time.


Function  Time (s)  Time (%)  Calls
dir       0.329     66.64     1
regexprep 0.049     9.99      5575
lstat     0.023     4.60      2787
localtime 0.021     4.33      2787
readdir   0.016     3.21      1
stat      0.012     2.51      600
strftime  0.012     2.47      2787
horzcat   0.011     2.21      2787


On my machine, testing with


tic; l = dir ('/usr/bin/'); toc


shows that Octave requires 167 milliseconds per 1,000 entries.  Or put another
way, about 6,000 filenames per second.  I believe that would generally be
sufficient for most purposes.

If more performance is required then either a JIT accelerator for loops in
m-files will be required, or the function will need to be converted to C++.

I tried the attached dir2 patch, but it no longer applies cleanly to the
development branch.




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?43327>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/