[Octave-bug-tracker] [bug #39479] 'pkg -forge list' takes 56.5 seconds

[Rik]

Follow-up Comment #18, bug #39479 (project octave):

I modified the code so that it might be inserted in the current pkg code.  One
issue is that it was returning the entire list of all packages and all package
versions which is over 600 entries.  I chopped off the version numbers with
another regexp and then used unique to get a list of true packages.


## Get the list of all packages
[html, r] = urlread
('https://sourceforge.net/projects/octave/files/Octave%20Forge%20Packages/Individual%20Package%20Releases');

if (! r)
  error ("pkg: failed to fetch list of packages from sourceforge.net");
endif

## Scrape the HTML itself
ptn = '<tr\s+title="(.*?gz)"\s+class="file';
[r, tok] = regexp (html, ptn, "start", "tokens");
if (isempty (r))
  error ("pkg: failed to parse list of packages from sourceforge.net");
endif

## Remove version numbers and produce unique list
files = cellstr (tok);
files = cellstr (regexp (files, '^.*?(?=-\d)', "match"));
files = unique (files);


However, the output is still not the same as 'pkg -forge list' which contains
69 entries, whereas this code produces 90 packages.  The difference is
possibly in deprecated or removed packages.  For example, "pt_br-1.0.9.tar.gz"
from 2010.

I don't think we should list packages like that.

It gets more complicated, but we could get the list of active packages from
the original URL 'https://octave.sourceforge.io/list_packages.php'.  And then
get the list of all packages with version numbers from the URL Andrew
provided.  And then use the names from the first to search in the second and
get the version number.  This seems pretty complicated though.

Why doesn't the original URL which is a php-generated page have the version
number in it?  Could we do that? 


(file #46621)
    _______________________________________________________

Additional Item Attachment:

File name: pkglist.m                      Size:0 KB
    <https://savannah.gnu.org/file/pkglist.m?file_id=46621>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?39479>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/