Re: URL for forge packages

[Carnë Draug]

On 18 August 2012 05:45, c. <address@hidden> wrote:
> On 17 Aug 2012, at 07:24, Carnë Draug wrote:
>> On 16 August 2012 13:14, Carnë Draug <address@hidden> wrote:
>>> Hi
>>>
>>> the current method to get the URL for packages when installing with
>>> the -forge flag is to parse the HTML of the package for the version
>>> and use it to generate a URL to download it. I see two problems with
>>> this:
>>>
>>> 1. if we are to change the format of the documentation page this will
>>> start failing
>>> 2. if we move the packages to another host (as has been discussed
>>> before), it will also fail
>>>
>>> While such changes are not planned to happen on OF soon, when they do
>>> it will mean that users using octave versions before the change won't
>>> be able to use the -forge flag. I think to prevent this before becomes
>>> an issie it's a good idea.  I would like to propose to have a simple
>>> text file on octave.org with this info. 1 line with package names and
>>> their URLs would suffice. This would also make the code to get
>>> packages from forge simpler since the parsing would also be simpler.
>>>
>>> Any comments?
>>
>> I gave this some more thought and here's the solution I propose. The
>> idea will be to have a text file in octave.org that is automatically
>> updated with a file I'll set up in SF. The following syntax for such
>> file would allow for other projects similar to Octave Forge (if and
>> when they appear) to also have their -project flag in pkg.
>>
>> =Forge http://path-text-file-with-list-of-forge-packages
>> package1-name package1-version package1-url
>> package2-name package2-version package2-url
>> package3-name package3-version package3-url
>> =ProjectX http://path-text-file-with-list-of-projectx-packages
>> package1-name package1-version package1-url
>>
>> Basically, the word after the equal would be the project and flag used
>> to get packages, followed by a URL for a text file that is
>> responsibility of that project. That text file would have just the
>> project, version, and url lines. Should be pretty simple for a perl
>> script to routinely check those URL's for changes and edit the file on
>> octave.org.
>>
>> I'd guess that XML would be better but having to parse them in Octave
>> is more complicated and I don't think we need that.
>>
>> Any comments? Can I start writing such scripts?
>>
>
> Carnë,
>
> I like the basic idea of your package list file and it seems a very good 
> strategy
> to allow smooth transition if we are to move away the current HTML pages from 
> Octave Forge.
>
> On the other hand there is one nice feature in the current setup that would 
> be nice to keep,
> i.e., it is automatically generated rather than manually maintained.
>
> So rather than setting up a format for a file to be maintained manually, I 
> would suggest to modify
> generate_package_html so that it generates a text file with the url of the 
> package to be downloaded
> and places it in the package docs directory, e.g. for package geometry there 
> should be a file
>
> http://octave.sourceforge.net/geometry/download_url.txt
>
> this would keep the release procedure more automatic and impose less work on 
> the site maintainer.
> the root of the url could be set as a variable in Octave so that changing 
> that could also be automated
> by issuing a command like
>
> forge_base_url ("http://www.new.forge.org/";)
>
> what do you think?

Of course. Just because I was suggesting a simple text file, doesn't
mean I meant to edit it by hand. The idea is to yes, have a script to
handle all of that. As Søren mentioned, using an octave file will even
makes things much easier for us. jwe suggested on IRC that we could
even use Octave's text format, no need to be a binary.

The only complication I see is the one that Thomas Yengst mentioned
about systems that are connected only to a private network. I see two
solutions. Either pkg needs an option to set the value where to look
for this file/database of URLs (I think that's best) or we recommend
people to change the source of their pkg installation. Those other
systems will want to change the values of the URLs for something
internal so using an octave format will make it more difficult for
them to update it automatically unless they also have Octave
installed. I don't know, I'm not very familiar on how things are
organized on those places. Anyone?

Carnë