[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv e
From: |
Daniel P . Berrangé |
Subject: |
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment |
Date: |
Fri, 13 May 2022 11:29:08 +0100 |
User-agent: |
Mutt/2.2.1 (2022-02-19) |
On Fri, May 13, 2022 at 09:35:23AM +0100, Daniel P. Berrangé wrote:
> On Thu, May 12, 2022 at 08:06:00PM -0400, John Snow wrote:
> > RFC: This is a very early, crude attempt at switching over to an
> > external Python package dependency for QMP. This series does not
> > actually make the switch in and of itself, but instead just switches to
> > the paradigm of using a venv in general to install the QEMU python
> > packages instead of using PYTHONPATH to load them from the source tree.
> >
> > (By installing the package, we can process dependencies.)
> >
> > I'm sending it to the list so I can show you some of what's ugly so far
> > and my notes on how I might make it less ugly.
> >
> > (1) This doesn't trigger venv creation *from* iotests, it merely prints
> > a friendly error message if "make check-venv" has not been run
> > first. Not the greatest.
>
> So if we run the sequence
>
> mkdir build
> cd build
> ../configure
> make
> ./tests/qemu-iotests/check 001
>
> It won't work anymore, until we 'make check-venv' (or simply
> 'make check') ?
>
> I'm somewhat inclined to say that venv should be created
> unconditionally by default. ie a plain 'make' should always
> everything needed to be able to invoke the tests directly.
>
> > (2) This isn't acceptable for SRPM builds, because it uses PyPI to fetch
> > packages just-in-time. My thought is to use an environment variable like
> > QEMU_CHECK_NO_INTERNET that changes the behavior of the venv setup
> > process. We can use "--system-site-packages" as an argument to venv
> > creation and "--no-index" as an argument to pip installation to achieve
> > good behavior in SRPM building scenarios. It'd be up to the spec-writer
> > to opt into that behavior.
>
> I think I'd expect --system-site-packages to be the default behaviour.
> We expect QEMU to be compatible with the packages available in the
> distros that we're targetting. So if the dev has the python packages
> installed from their distro, we should be using them preferentially.
>
> This is similar to how we bundle slirp/capstone/etc, but will
> preferentially use the distro version if it is available.
AFAICT from testing it, when '--system-site-packages' is set
for the venv, then 'pip install' appears to end up being a
no-op if the package is already present in the host, but
installs it if missing.
IOW, if we default to --system-site-packages, but still
also run 'pip install', we should "do the right thing".
It'll use any distro packages that are available, and
augment with stuff from pip. In the no-op case, pip will
still try to consult the pipy servers to fetch version
info IIUC, so we need to be able to stop that. So I'd
suggest a --python-env arg taking three values
* "auto" (the default) - add --system-site-packages, but
also run 'pip install'. Good for developers day-to-day
* "system" - add --system-site-packages but never run
'pip install'. Good for formal package builds.
* "pip" - don't add --system-site-packages, always run
'pip install'. Good for testing compatibility with
bleeding edge python versions, or if explicit full
independance from host python install is desired.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
- Re: [RFC PATCH 6/9] tests: add check-venv as a dependency of check and check-block, (continued)
[RFC PATCH 5/9] tests: use tests/venv to run basevm.py-based scripts, John Snow, 2022/05/12
[RFC PATCH 8/9] iotests: fix source directory location, John Snow, 2022/05/12
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment, Daniel P . Berrangé, 2022/05/13
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment, Kevin Wolf, 2022/05/13
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment, John Snow, 2022/05/13
Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment, Paolo Bonzini, 2022/05/13