Re: Questions regarding Python packaging

[Maxim Cournoyer]

Hello,

Tanguy LE CARROUR <tanguy@bioneland.org> writes:

> Excerpts from Tanguy LE CARROUR's message of January 22, 2021 9:38 am:
>> Excerpts from Tanguy LE CARROUR's message of January 6, 2021 4:32 pm:
>>> Excerpts from Lars-Dominik Braun's message of January 5, 2021 11:25 am:
>>>>> So, I've tried packaging `python-keyring` with those two…
>>>>> 
>>>>> `pep517` keeps on trying to download dependencies, which won't work.
>>>>> 
>>>>> `build` crashes with "ZIP does not support timestamps before 1980",
>>>>> which, I guess is related to the fact that everything in the store is
>>>>> timestamped to January 1st 1970.
>>>> have you been looking into a python-build-system using `build`[1]? I’ve
>>>> had the same issue with egg versions set to 0.0.0 and think in the long
>>>> run moving to a PEP 517-style build is the way forward.
>>> 
>>> I agree! Unfortunately, I haven't had much time (so far) to work on it! :-(
>>> 
>>> I'll revive the thread as soon as I've made progress…
>> 
>> Done! :-)
>> I've eventually succeeded in ("properly") packaging a software managed
>> with Poetry. And I've learned quite a lot on the way!
>
> Just for the sake of having it documented somewhere, here is the
> relevant part of the `guix.scm` file:
>
> ```
> (define-public nowty
>   (package
>     (name "nowty")
>     (version (string-append %pyproject-version "+" %git-commit))
>     (source (local-file %source-dir #:recursive? #t))
>     (build-system python-build-system)
>     (arguments
>      `(#:phases
>        (modify-phases %standard-phases
>         (replace 'build
>          (lambda* (#:key #:allow-other-keys)
>            (substitute* "pyproject.toml"
>             (((string-append "version = \"" ,%pyproject-version "\"" ))
>              (string-append "version = \"" ,version "\"")))))
>         (replace 'install
>          (lambda* (#:key outputs #:allow-other-keys)
>           (let ((out (assoc-ref outputs "out")))
>            (invoke "python" "-m" "pip" "install"
>                    "--no-dependencies" "--no-build-isolation" "--prefix" out 
> "."))))
>         (replace 'check
>          (lambda* (#:key inputs outputs #:allow-other-keys)
>           (add-installed-pythonpath inputs outputs)
>           (invoke "python" "-m" "invoke" "test.unit"))))))
>     (native-inputs
>      `(("python-invoke" ,python-invoke-patched)
>        ("python-mamba" ,python-mamba)
>        ("python-poetry-core" ,python-poetry-core)
>        ("python-robber" ,python-robber)
>        ("python-termcolor" ,python-termcolor)))
>     (propagated-inputs
>      `(("python-mpd2" ,python-mpd2)
>        ("python-typer" ,python-typer)))
>     (synopsis "Music notification daemon for MPD")
>     (description "A music notification daemon that monitors songs being 
> played by MPD
> and displays notifications with the song's details.")
>     (home-page "http://projects.bioneland.org/nowty";)
>     (license license:gpl3+)))
> ```

As another PEP 517 package build example, you may want to look at
python-isort on core-updates (commit
812a2931de553d12c01b0a4d53d03613b09adaaf).  Here's the definition:

--8<---------------cut here---------------start------------->8---
(define-public python-isort
  (package
    (name "python-isort")
    (version "5.7.0")
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             ;; Tests pass only from the Github sources
             (url "https://github.com/timothycrosley/isort";)
             (commit version)))
       (file-name (git-file-name name version))
       (modules '((guix build utils)))
       (snippet '(for-each delete-file (find-files "." "\\.whl$")))
       (sha256
        (base32
         "0phq4s911mjjdyr5h5siz93jnpkqb2qgphgcfk6axncgxr8i7vi1"))))
    (build-system python-build-system)
    (arguments
     `(#:phases
       (modify-phases %standard-phases
         ;; A foretaste of what our future python-build-system will need to
         ;; do.
         (replace 'build
           (lambda _
             (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
         (replace 'install
           (lambda* (#:key outputs #:allow-other-keys)
             (let ((out (assoc-ref outputs "out"))
                   (whl (car (find-files "dist" "\\.whl$"))))
               (invoke "pip" "--no-cache-dir" "--no-input"
                       "install" "--no-deps" "--prefix" out whl))))
         (add-after 'install 'install-example-plugins
           (lambda* (#:key outputs #:allow-other-keys)
             (let ((out (assoc-ref outputs "out")))
               ;; Patch to use the core poetry API.
               (substitute* '("example_isort_formatting_plugin/pyproject.toml"
                              "example_shared_isort_profile/pyproject.toml")
                 (("poetry>=0.12")
                  "poetry-core>=1.0.0")
                 (("poetry.masonry.api")
                  "poetry.core.masonry.api"))
               ;; Build the example plugins.
               (for-each (lambda (source-directory)
                           (invoke "python" "-m" "build" "--wheel"
                                   "--no-isolation" "--outdir=dist"
                                   source-directory))
                         '("example_isort_formatting_plugin"
                           "example_shared_isort_profile"))
               ;; Install them to temporary storage, for the test.
               (setenv "HOME" (getcwd))
               (let ((example-whls (find-files "dist" "^example.*\\.whl$")))
                 (apply invoke "pip" "--no-cache-dir" "--no-input"
                        "install"  "--user" "--no-deps" example-whls)))))
         (replace 'check
           (lambda* (#:key tests? inputs outputs #:allow-other-keys)
             (when tests?
               (let ((bin (string-append (assoc-ref outputs "out") "/bin")))
                 (setenv "PATH" (string-append (getenv "PATH") ":" bin)))
               (add-installed-pythonpath inputs outputs)
               (invoke "pytest" "-vv" "tests/unit/"
                       "--ignore=tests/unit/test_deprecated_finders.py")))))))
    (native-inputs
     `(("git" ,git-minimal)
       ("python-black" ,python-black)
       ("python-colorama" ,python-colorama)
       ("python-hypothesmith" ,python-hypothesmith)
       ("python-libcst" ,python-libcst-minimal)
       ("python-poetry-core" ,python-poetry-core)
       ("python-pylama" ,python-pylama)
       ("python-pypa-build" ,python-pypa-build)
       ("python-pytest-mock" ,python-pytest-mock)
       ("python-pytest" ,python-pytest)))
    (home-page "https://github.com/PyCQA/isort";)
    (synopsis "Python utility/library to sort python imports")
    (description "@code{python-isort} is a python utility/library to sort
imports alphabetically, and automatically separated into sections.  It
provides a command line utility, a python library and plugins for various
editors.")
    (license license:expat)))
--8<---------------cut here---------------end--------------->8---

That's an ad-hoc solution pending an update to our Python build system
phases.

HTH!

Maxim