Re: Avoiding PYTHONPATH - latest?

[Phil]

I should have added the pyvenv.cfg file should look like this:

$ cat pyvenv.cfg 
home = /replace/with/out/variable/in/guix
include-system-site-packages = true
version = replace.with.guix.version.variable

We include system packages to ensure we take site-packages from both the
Python 3.8 store's and the profile's site-packages.

----

One thing I have noticed is that if you want to further nest a venv
(using the regular: python3 -m venv) inside the setup below - that will
NOT work if you include --system-site-packages in the nested venv.  The
profile's site-packages are not included, because it would seem that
python does not respect the outer pyvenv.cfg in nested venvs.

If this can't be solved it pours cold water on the idea somewhat.


Phil writes:

> Hi all,
>
> I've been having an argument with myself over the last 4 days about if
> Guix's use of PYTHONPATH is a necessary evil or avoidable on a foreign OS.
>
> I've found references to a similar discussion last year, and reference
> to using a 'fake virtual environment' (does anyone have a reference to
> the other thread referenced where fake venvs are demonstrated as not
> working):
>
> https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00204.html
> https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00221.html
>
> I had what I think is a similar idea over the weekend and tried it out.
>
> Reading the mechanics of venvs as per PEP:
> https://www.python.org/dev/peps/pep-0405
>
> I decided to see if I added a fake pyvenv.cfg to the python.scm package
> could allow for us to drop the use of PYTHONPATH completely.
>
> Now the results are very preliminary, and I am essentially replacing
> one hack with another (I'm not creating a venv after-all) - it does seem
> to work just adding this to the python-3.8's phases: 
>
> (add-after 'install 'add-pyvenv-cfg
>            (lambda* (#:key outputs #:allow-other-keys)
>                      ;; guix-daemon --build-users-group=guixbuild 
> --chroot-directory=/path/to/guix_chroot_extra
>                      (let ((out (assoc-ref outputs "out")))
>                        (display (string-append "\nOutput Directory: \n" out 
> "\n"))
>                        (install-file "/path/to/guix_chroot_extra/pyvenv.cfg" 
> out)
>                        (substitute* (string-append out "/pyvenv.cfg")
>                                     (("^home.*") (string-append "home = " out 
> "\n"))
>                                     (("^version.*") (string-append "version = 
> 3.8.2\n"))))))
>
> The native-search-paths can then have PYTHONPATH removed (I emulated
> this by unset'ing it after creating a profile with the modified version
> of the python package).
>
> Starting python and looking at sys.path I could see this worked, both in
> the cases where lib and bin symlink their counterpart
> directories in the store, and also work when real lib and bin
> directories in the profile contain symlinks to the store at the file level.
>
> It seems so simple I'm sure it's been tried and found to misbehave, as I
> hands-up accept it's misuse venv's funtionality (using pyvenv.cfg
> without setting VIRTUAL_ENV is not defined behaviour).
>
> But the current PYTHONPATH implementation immediately runs into problems
> when you use venvs for development, via the cannonical "pip install -e
> .".  The use of PYTHONPATH means that venvs are not created pure, they
> inherit Guix's python env which is the exact opposite to what any
> non-Guix python developer would expect.
>
> Ironically when sharing environment setup responsibilities between
> profiles and venvs like this, inheriting the profile's PYTHONPATH is
> often useful - but it's still unexpected.
>
> The PYTHONPATH, if I've understood correctly, is for user-level tweaks,
> so hijacking it for system use may cause issues.
>
> I was wondering what the current thoughts were on how Guix could
> workaround using PYTHONPATH to set base system paths?
>
> Thanks,
> Phil.