Re: Reproducing a Python project environment (using guix inferiors)

[zimoun]

Hi,

On Sat, 28 Nov 2020 at 18:09, branjam4@gmail.com wrote:

> Some time before I started using guix I worked through Python examples
> within this book[1] for school. There were some differences in the
> packages I installed and the listed requirements for the book (pseudo
> manifest at [2]) which made things difficult for me as a (then) new
> programmer.

Cool!  Thank you for your feedback.


> Believing guix could more faithfully/cleanly reproduce the environment
> from the book than my former approach using pip, I:
> + imported/wrote definitions for packages not in guix

Usually, this is done via the option “--load-path” or Custom Channels.

<https://guix.gnu.org/manual/devel/en/guix.html#Creating-a-Channel>

> + wrote inferiors for older versions of packages that were in guix.

You might be interested by this Channel:

  <https://gitlab.inria.fr/guix-hpc/guix-past>


> I'm not sure what the rhyme or reason is, but most of my inferior
> attempts failed with this backtrace[3]. A couple were successful, as you
> can see in this table[4].

I am not sure to understand what you are naming “inferior” here.  From
my experience, one simple way to start is:

  guix time-machine --commit=<old> \
       -- build -L <path/to/local/packages> <your-package>

where <old> is an old Guix commit providing bunch of dependencies used
by <your-package> of interest defined in the folder
<path/to/local/packages>.

A channels.scm file with the option ’--channels/-C’ seems even better.


> But how would I know what a "good" guix inferior commit is for all of
> these packages?

Let’s fix ideas and let’s say you are interested by the package
python-foo at the version 1.2.3.  This package depends on python-bar at
version x.y.z available at commit range 12345-67890 and on python-baz at
version m.n available at commit range ABCDE-FGHIJ.

Then there is no commit where python-bar *and* python-baz are both at
the correct versions (resp. x.y.z and m.n.).  In other words, the 2
commit ranges 12345-67890 and ABCDE-FGHIJ do not overlap.

Is it your use case?

BTW, a tool to find the range of commit where one specific version is
available is the Data Service.  Give a look at:

<https://data.guix.gnu.org/repository/1/branch/master/package/python-scipy/output-history>


> Would using the definitions instead of asking guix to reproduce
> five different worlds for one package each be an anti-pattern?
> Is my issue related to being on a foreign distro, thus Guix System users
> wouldn't know much about this problem?

Being on a foreign distro does not change.  The main difference between
between foreign and native is only about services, not packages.


> Even though I'm mentioning this specific frustration, I enjoyed the
> learning process getting to this point and want to express my
> appreciation. Despite interacting with clojure and emacs lisp for at
> least a year, guix/guile finally got me to use quoting, let, and lambda
> in a non-trivial way. Thus the otherwise tedious work of generating five
> different channel/inferior definitions as per the example in the manual
> was actually a breeze!

Cool!


> [1] Complex Network Analysis Using Python, by Dmitry Zinoviev

I will try to give a look if my library has this reference.


> [2] For some of the versioned packages, I commented the newest commit
> prior to guix bumping that package, which I would use for an inferior.
> #+BEGIN_SRC scheme
> (specifications->manifest ;inferior commit notes
>  '("python" ; not trying to build this from scratch.
>    "python-matplotlib@2.0.2" ;7e06086522
>    "python-nltk" ;need 3.2.5
>    "python-pandas@0.22.0" ;ce2cfcabfc
>    ;"python-wikipedia@1.4.0" need to guix import
>    ;"python-toposort@1.5" need to guix import
>    "python-networkx@2.1" ;269f100330
>    ;"python-community@0.10" need to guix import
>    "python-numpy@1.13.3" ;4d6ed794dd
>    "python-pygraphviz" ;want 1.3 but only 1.5 available
>    ;;may need graphviz-dev as well
>    "python-scipy@1.0.1" ;02ddafef55
>    ;"python-louvain@0.14" need to guix import
>    ;need to install the author's custom modules))
> #+END_SRC
> I used the above as a guide while doing the actual imports/inferior
> work.

I have tried to write a script for my personal needs running via “guix
repl”.  It fetches the JSON file from the Data Service, which give the
commit range for the required version per package.  Then I have not
finished it (yet) but the idea was to be able to spot out the commit
providing all the packages at the required versions, i.e., the
intersection.  Pieces of the necessary material is in guix/git.scm.

<https://git.savannah.gnu.org/cgit/guix.git/tree/guix/git.scm#n444>


> [3]
> #+begin_example scheme
> In current input:
>      12:3 12 (_)
> In guix/store.scm:

[...]

>     619:8  4 (_ #(#(#(#<directory (build-self) 1842aa0>) "/gnu…" …) …))
>    626:19  3 (_ #(#(#(#<directory (build-self) 1842aa0>) "/gnu…" …) …))
>     155:9  2 (_ #(#(#(#<directory (build-self) 1842aa0>) "/gnu…" …) …))
>    223:20  1 (proc #(#(#(#<directory (build-self) 1842aa0>) "/…" …) …))
> In unknown file:
>            0 (%resolve-variable (7 . %guix-register-program) #<direc…>)
> #+end_example

What did you run?  The command and the file(s)?



> [4]
> | package of interest | guix commit  | status |
> |---------------------+--------------+--------|
> | python-matplotlib   | "7e06086522" | bad    |
> | python-pandas       | ce2cfcabfc   | bad    |
> | python-networkx     | 269f100330   | good   |
> | python-numpy        | 4d6ed794dd   | bad    |
> | python-scipy        | 02ddafef55   | good   |

What do you mean by “bad”?
Does it mean:

 1. the package does not build
 2. the package is failing
 3. else

?


All the best,
simon