Re: Replacing python venv - environment or profile?

[zimoun]

Hi Phil,

On Tue, 24 Nov 2020 at 17:22, Phil <phil@beadling.co.uk> wrote:

> I'm new the Guix and considering adopting it as an expansion to my
> current use of Python virtual environments in development and
> production, to include other non-Python packages too.

Nice!


> I've read the manual/cookbook and I get the gist of the tool.  One thing
> I'm getting a bit stuck on is the appropriate use of environment vs
> profile; different sources give slightly different takes on the theme, and I
> want to make sure my use is correct from the get-go.

Initially, coming from Python terminology, I was confused too.

Well roughly, an environment is a temporary profile.  So, in the Python
terminology, the “environment” corresponds to the Guix profile.  And the
Guix environment does not have an equivalent in the Python terminology,
AFAIK.


> I'll be using Guix on a foreign OS, I suspect the answers might be
> slightly different if I was able to adopt the Guix System OS, but for
> now I can't.

No, it should change nothing.  IIUC.

>
> As well as the manual cookbook I came across this blog:
> https://trivialfis.github.io/linux/2018/06/10/Using-guix-for-development.html

Thanks for the pointer.  I have not read it yet.


> I understand that I can use the --pure switch to ensure no pollution

To understand the difference between ’--pure’ or not:

       $ guix environment --ad-hoc python
 [env] $ echo $PATH
vs
       $ guix environment --ad-hoc python --pure
 [env] $ echo $PATH

And this is done for all the environment variables.


> This is great - but the manual points out that none of the
> installed packages are marked such that they avoid garbage collection.

The Guix environment are temporary.  Compare:

a  $ guix environment --ad-hoc hello -- hello
b  $ guix gc --list-dead | grep hello
c  $ guix gc --list-live | grep hello

d  $ guix install hello
e  $ guix gc --list-dead | grep hello
f  $ guix gc --list-live | grep hello

The a create a temporary profile (environment) containing the package
hello and then the program hello is run inside.

The b lists all the items in the store that are not used by any profile,
so they could be garbage collected because nothing need them.  The c
lists all the live items, i.e, all the item used by one or more profile,
so they cannot be garbage collected.

The d installs hello in the default profile (~/.guix-profile), so now
the e shows nothing because one profile uses it and the f shows.

Does it make sense?


The next point is that you can create any profile (as the Python
environment).  For example,


  guix install python python-mypy -p ~/my-first-profile


The big difference is “guix pull”.  Well, imagine this sequence:

  guix environment --ad-hoc python
  [..work..]
  guix pull
  guix environment --ad-hoc python

Now, the first Python is not necessary the exact same than the second
one.  Considering the sequence,

  guix install python -p /my-profile
  [..work..]
  guix pull
  eval $(guix package --search-paths=prefix -p ~/my-profile)

The Python inside your profile ~/my-profile has not changed.


> So I came up with something like the below - manifests do seem to be an
> exact analogy to Python requirements.txt files:

Yes, manifest.scm is equivalent to requirements.txt.  The only
difference is that power because it is possible to write Scheme inside
this file and so apply package transformation, create on-the-fly package
variant, etc.  Well, another story. :-)


> $ cat manifest.scm 
> (specifications->manifest
>   '("coreutils"
>     "emacs@27.1"
>     "python"
>     "python-pytest"
>     "python-coverage"
>     "python-pytest-cov"
>     "python-black"
>     "python-mypy"
>     "python-flake8"))
> $ guix environment --pure --manifest=manifest.scm --root=./test-profile

I am not convinced you need the --root at first.


> This creates a profile as a side-effect, I think.

This create a environment, not a profile.  For a profile, you should do:

  guix package -m manifest.scm -p ./test-profile


> So to my mind Guix profiles are more like Python virtual environments with
> the with the non-standard --system-site-packages switch than the
> classic

Yes.

> use of Python virtual environment?  Useful, but different.  The 'guix
> environment' is accidentally providing a closer parallel.

AFAIK, the Guix environment has no equivalent.  For example, you can
create a Guix environment in a container; --pure on steroids. :-)


> Finally.... I get to my question!  So in light of the above (assuming I
> haven't missed the point completely!), what is the canonical way of
> isolating a virtual environment using Guix on a foreign OS
> installation?

Profile and then the correct --search-paths.

But if you want crazy isolation with container, you can do:

  guix describe -f channels > my-channels.scm

then any point in time, you will have the exact same environment with:

  guix time-machine -C my-channels -- environment -C -m my-manifest.scm


> Last point - ultimately as well as using Guix to provide a consistent
> profile/environment to do Python work in, I'd ultimately
> like to hook Guix up to Jenkins so that it can package and deploy a
> repo to a

That’s a bigger story…

> production server.  This probably involves setting-up a private Guix
> Channel and importing built wheels, and to then pull the new Guix package 
> from the
> production server.  Ultimately tho I'd want the same thing - a pure
> environment in production which has access to exactly and only the
> software as described in the manifest.  This feels like it should be
> possible too, but again, just looking for any initial guidance on which 
> features
> of Guix to use to do this.

…so let’s discuss it once you are comfortable with Guix on your local
machine. ;-)


All the best,
simon