Re: About packaging documentation

[Zelphir Kaltstahl]

Hello Ricardo!

On 3/16/21 11:03 PM, Ricardo Wurmus wrote:
> Hi Zelphir,
>
>> Hello Ricardo!
>>
>> On 3/15/21 4:43 PM, Ricardo Wurmus wrote:
>>> Hi Zelphir,
>>>
>>>> https://guix.gnu.org/cookbook/en/html_node/Packaging-Tutorial.html
>>>> is a good
>>>> start. It has even got cookbook in the name. Definitely the right idea to
>>>> complement the general documentation. I am aware of its existence.
>>>> Unfortunately, I could not get the basic approach of the "Hello World 
>>>> package"
>>>> working for my package, even though it is pure guile, no other library 
>>>> required
>>>> and no FFI or anything. I would have preferred not having to go through 
>>>> all the
>>>> autotools stuff, and to have this simple way working for my package. 
>>>> Perhaps I
>>>> did something slightly wrong. I do not know. Someone mentioned on the 
>>>> guile user
>>>> mailing list, that this is all that should be needed for a pure guile 
>>>> package.
>>>> Perhaps it can be updated?
>>> Could you share the code you’re trying to package?  Perhaps it will
>>> become clearer to us what you would like to see changed in the cookbook
>>> — and perhaps it will become clearer to you how packaging for Guix works.
>> Yes, I'll link it:
>>
>> https://notabug.org/ZelphirKaltstahl/guile-fslib/commit/eacdbb5ee9e30413392908d9e3988e30e9411aa7
>> <https://notabug.org/ZelphirKaltstahl/guile-fslib/commit/eacdbb5ee9e30413392908d9e3988e30e9411aa7>
>>
>> Or:
>>
>> https://notabug.org/ZelphirKaltstahl/guile-fslib/src/0.2.0
> Excellent.
>
> Here’s the package (I dumped this in (gnu packages guile-xyz)):
>
> --8<---------------cut here---------------start------------->8---
> (define-public guile-fslib
>   (package
>     (name "guile-fslib")
>     (version "0.2.0")
>     (source
>      (origin
>        (method git-fetch)
>        (uri (git-reference
>              (url "https://notabug.org/ZelphirKaltstahl/guile-fslib/";)
>              (commit version)))
>        (file-name (git-file-name name version))
>        (sha256
>         (base32
>          "118d84p443w7hrslv8hjyhgws631ia08mggiyklkmk0b9plfdsvz"))))
>     (build-system guile-build-system)
>     (inputs
>      `(("guile" ,guile-3.0)))
>     (home-page "https://notabug.org/ZelphirKaltstahl/guile-fslib";)
>     (synopsis "File system utilities")
>     (description
>      "This package contains like super cool file system utilities and stuff.
> It's really good and so easy to install!")
>     (license license:agpl3+)))
> --8<---------------cut here---------------end--------------->8---
>
> I only gave it a quick test like this:
>
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix environment --ad-hoc guile  guile-fslib -- guile
> […]
> GNU Guile 3.0.5
> Copyright (C) 1995-2021 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> ,use (file-system)
> scheme@(guile-user)> file-size-in-bytes
> $1 = #<procedure file-size-in-bytes (path)>
> scheme@(guile-user)> 
> --8<---------------cut here---------------end--------------->8---
>
> Seems to work.
>
> And this is what was installed:
>
> --8<---------------cut here---------------start------------->8---
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/logging.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/test
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/test/test-fslib.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/test/test-list-utils.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/test/test-string-utils.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/fslib.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/file-reader.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/list-utils.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/file-system.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/lib/guile/3.0/site-ccache/string-utils.go
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/file-system.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/file-reader.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/test
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/test/test-string-utils.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/test/test-fslib.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/test/test-list-utils.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/string-utils.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/fslib.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/list-utils.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/guile/site/3.0/logging.scm
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/doc
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/doc/guile-fslib-0.2.0
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/doc/guile-fslib-0.2.0/todo.org
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/doc/guile-fslib-0.2.0/LICENSE
> /gnu/store/2k3x0j6mvypjagam39dx89fd5yan7a3y-guile-fslib-0.2.0/share/doc/guile-fslib-0.2.0/README.md
> --8<---------------cut here---------------end--------------->8---
>
> One thing to note is that Guix didn’t run the tests, because it doesn’t
> know how to (there’s no generic way to run tests for Guile packages, so
> the build system doesn’t try).
>
> This can be changed by adding an “arguments” field to the package
> definition that adds a phase to run the tests.

First of all: Thanks again for providing the example definition of a package
using the guile-build-system. This might well be the missing piece in the whole
puzzle.

I've looked at the cookbook again. The following things or aspects of the
cookbook are not clear to me or cause confusion:

(1) The cookbook does not even mention the guile-build-system. There is no
example of it in the cookbook and no description of what a project needs to
prepare or do, so that it can use the guile-build-system. Reading the cookbook
one will not even know about the existence of guile-build-system. I checked, I
searched the whole page at https://guix.gnu.org/cookbook/en/guix-cookbook.html
<https://guix.gnu.org/cookbook/en/guix-cookbook.html> for "guile-build-system".
Zero hits. To be fair, the cookbook links
to https://guix.gnu.org/manual/en/guix.html#Build-Systems
<https://guix.gnu.org/manual/en/guix.html#Build-Systems>, wherein there is a
description of `guile-build-system`. There it says, what the guile-build-system
is for, but no example is given.

I guess this is, because the cookbook tries to be generic in a way, to apply to
all languages, if only one uses autotools, as those are applicable to all
languages. That is actually a good idea. I think, that build system specific
examples would help people, who might not need the power of autotools, but for
example have a pure GNU Guile project.

This is probably, where I went wrong, when I tried to follow the cookbook months
ago. I probably tried using gnu-build-system and failed, because I had no
autotools infrastructure set up in my project.

(2) The hello package example is using the gnu-build-system. I guess that means,
that Guix will go through the usual autotools steps, after extracting a tarball
or cloning a repository. However, then it would seem, that the success of the
installation of the package largely hinges on what is defined in the autotools
related files, like Makefile and such, which are part of the downloaded tar.
Those however, are not described in the cookbook. If one is to create a package,
which makes use of the gnu-build-system, then a description of how to get this
done would be good to have. People have done it before, but documentation does
not capture it. A good guide about this would have saved me a month of trying
things, mailing list interaction, and write such guide myself. On the other
hand, I would probably have learned less in the process.

Getting back to the package definition example:

Then there is another confusion: The definition you have written – Is that for
editing the `guile.scm` in the Guix source tree, or is that for usage with `guix
package --install-from-file=somename.scm` as shown for the hello package in the
cookbook?

My guess is, that it is for editing the Guix sources, because using it for `guix
package --install-from-file` will not work, as the imports are missing and there
is no "package value" returned at the end. I try to add those parts, but then
more stuff is missing:

~~~~package definition~~~~
(use-modules (guix packages)
             (guix download)
             (guix build-system gnu)
             ((guix licenses) #:prefix license:))

(define-public guile-fslib
  (package
    (name "guile-fslib")
    (version "0.2.0")
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             (url "https://notabug.org/ZelphirKaltstahl/guile-fslib/";)
             (commit version)))
       (file-name (git-file-name name version))
       (sha256
        (base32
         "118d84p443w7hrslv8hjyhgws631ia08mggiyklkmk0b9plfdsvz"))))
    (build-system guile-build-system)
    (inputs
     `(("guile" ,guile-3.0)))
    (home-page "https://notabug.org/ZelphirKaltstahl/guile-fslib";)
    (synopsis "File system utilities")
    (description
     "This package contains like super cool file system utilities and stuff.
It's really good and so easy to install!")
    (license license:agpl3+)))

guile-fslib
~~~~

~~~~cli interaction~~~~
$ guix package --install-from-file=guix.scm

/home/user/dev/guile/guile-fslib/guix.scm:18:0: error: git-fetch: unbound 
variable
hint: Did you forget a `use-modules' form?
~~~~

So I looked at the guix sources and found git-fetch at
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/git-download.scm
<https://git.savannah.gnu.org/cgit/guix.git/tree/guix/git-download.scm>.

I guess the import should be `(guix git-download)`?

Then the next one is missing: `guile-build-system`. -> (guix build-system guile)

Then `guile-3.0` is missing. -> (gnu packages guile)

After adding these the whole file looks as follows:

~~~~
(use-modules (guix packages)
             (guix download)
             (guix git-download)
             (guix build-system gnu)
             (guix build-system guile)
             (gnu packages guile)
             ((guix licenses) #:prefix license:))

(define-public guile-fslib
  (package
    (name "guile-fslib")
    (version "0.2.0")
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             (url "https://notabug.org/ZelphirKaltstahl/guile-fslib/";)
             (commit version)))
       (file-name (git-file-name name version))
       (sha256
        (base32
         "118d84p443w7hrslv8hjyhgws631ia08mggiyklkmk0b9plfdsvz"))))
    (build-system guile-build-system)
    (inputs
     `(("guile" ,guile-3.0)))
    (home-page "https://notabug.org/ZelphirKaltstahl/guile-fslib";)
    (synopsis "File system utilities")
    (description
     "This package contains like super cool file system utilities and stuff.
It's really good and so easy to install!")
    (license license:agpl3+)))

guile-fslib
~~~~

(I know, I might not need some of the imports remaining, like the
`gnu-build-system` from `(guix build-system gnu)`, as I am then using the
`guile-build-system`.)

I think an example with `guile-build-system` would be good to have in the
cookbook. The other docs (almost?) exclusively mention gnu-build-system only as
well.

If this definition of a package works using `guix package
--install-from-file=somename.scm`, is it safe to assume, that it will work when
putting that definition inside `guile.scm` in the GNU Guix sources? If it is not
safe to assume this, what are the things, that make it unsafe to assume so?

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl