[Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow

[Rik]

Follow-up Comment #7, bug #60387 (project octave):

I don't think cURL is doing anything fancy.  This might be a case where we
want to intervene and parse the output of the single dir() transaction.  From
'help ftp', most of the commands are one-shot transactions:


     ascii       Set transfer type to ascii
     binary      Set transfer type to binary
     cd          Change remote working directory
     close       Close FTP connection
     delete      Delete remote file
     dir         List remote directory contents
     mget        Download remote files
     mkdir       Create remote directory
     mput        Upload local files
     rename      Rename remote file or directory
     rmdir       Remove remote directory


The notable exceptions are mget, mput, and dir which might iterate over an
input list.

The first code C++, called by @ftp class, is in libinterp/corefcn/__ftp__.cc. 
The interesting bit for the dir command is


DEFMETHOD (__ftp_dir__, interp, args, nargout,
           doc: /* -*- texinfo -*-
@deftypefn {} {} __ftp_dir__ (@var{handle})
Undocumented internal function
@end deftypefn */)
{
  if (args.length () != 1)
    error ("__ftp_dir__: incorrect number of arguments");

  octave::url_handle_manager& uhm = interp.get_url_handle_manager ();

  octave::url_transfer url_xfer = uhm.get_object (args(0));

  if (! url_xfer.is_valid ())
    error ("__ftp_dir__: invalid ftp handle");

  octave_value retval;

  if (nargout == 0)
    url_xfer.dir ();
  else
    {
      string_vector sv = url_xfer.list ();
      octave_idx_type n = sv.numel ();

      if (n == 0)
        {
          string_vector flds (5);

          flds(0) = "name";
          flds(1) = "date";
          flds(2) = "bytes";
          flds(3) = "isdir";
          flds(4) = "datenum";

          retval = octave_map (flds);
        }
      else
        {
          octave_map st;

          Cell filectime (dim_vector (n, 1));
          Cell filesize (dim_vector (n, 1));
          Cell fileisdir (dim_vector (n, 1));
          Cell filedatenum (dim_vector (n, 1));

          st.assign ("name", Cell (sv));

          for (octave_idx_type i = 0; i < n; i++)
            {
              time_t ftime;
              bool fisdir;
              double fsize;

              url_xfer.get_fileinfo (sv(i), fsize, ftime, fisdir);

              fileisdir (i) = fisdir;
              filectime (i) = ctime (&ftime);
              filesize (i) = fsize;
              filedatenum (i) = double (ftime);
            }

          st.assign ("date", filectime);
          st.assign ("bytes", filesize);
          st.assign ("isdir", fileisdir);
          st.assign ("datenum", filedatenum);

          retval = st;
        }
    }

  return retval;
}


First, this is some very old code.  It checks for the correct number of input
arguments, but this is a completely internal function.  It is up to the caller
to get it right and such a check should be removed.  You can also see that
this is old in that the indexing style doesn't cuddle the parenthis for
variables like fileisdir.  Again, just an indication that this is old.

As you can see, if nargout is 0 then this just calls url_xfer.dir() which is
presumably efficient.  Otherwise, it calls url_xfer.get_fileinfo () for each
file.  Presumably, that is a slow call.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?60387>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/