[Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow

[Rik]

Follow-up Comment #9, bug #60387 (project octave):

This code seems really, really complicated for doing what it does.  Actions
start by calling a method function of an old-style classdef object in @ftp
dir.  From there it jumps to C++ function in libinterp/corefcn/__ftp__.cc. 
Functions there call other functions in url_manager.cc.  Eventually, some
calls seem to be split between Octave's own iostream class and url_transfer
class which is in liboctave/utils.  And over in liboctave the code is split
between a class that just holds a rep pointer and the base class which
actually does the work.

Anyways, at the heart of the dir loop are calls to get_fileinfo which is
located in liboctave/util/url-transfer.cc.  The function is


    void get_fileinfo (const std::string& filename, double& filesize,
                       time_t& filetime, bool& fileisdir)
    {
      std::string path = pwd ();

      m_url = "ftp://"; + m_host_or_url + '/' + path + '/' + filename;
      SETOPT (CURLOPT_URL, m_url.c_str ());
      SETOPT (CURLOPT_FILETIME, 1);
      SETOPT (CURLOPT_HEADERFUNCTION, throw_away);
      SETOPT (CURLOPT_WRITEFUNCTION, throw_away);

      // FIXME
      // The MDTM command fails for a directory on the servers I tested
      // so this is a means of testing for directories.  It also means
      // I can't get the date of directories!

      perform ();
      if (! good ())
        {
          fileisdir = true;
          filetime = -1;
          filesize = 0;

          return;
        }

      fileisdir = false;
      time_t ft;
      curl_easy_getinfo (m_curl, CURLINFO_FILETIME, &ft);
      filetime = ft;
      double fs;
      curl_easy_getinfo (m_curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &fs);
      filesize = fs;

      SETOPT (CURLOPT_WRITEFUNCTION, write_data);
      SETOPT (CURLOPT_HEADERFUNCTION, 0);
      SETOPT (CURLOPT_FILETIME, 0);
      m_url = "ftp://"; + m_host_or_url;
      SETOPT (CURLOPT_URL, m_url.c_str ());

      // The MDTM command seems to reset the path to the root with the
      // servers I tested with, so cd again into the correct path.  Make
      // the path absolute so that this will work even with servers that
      // don't end up in the root after an MDTM command.
      cwd ('/' + path);
    }


This is performing a large number of cURL operations and is likely where the
slow down lies.  The 'cwd' command alone is not costless.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?60387>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/