Re: [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements

[Pavel Dovgalyuk]

Hi, Cleber!

What about record/replay tests from 25.06?

On 24.09.2021 21:54, Cleber Rosa wrote:
> This is a collection of patches for the Acceptance Tests to leverage
> some of the features of Avocado 91.0.  With the Avocado version bump
> by itself, there would be a change in the default "test runner"
> implementation that Avocado uses, from the one simply known as
> "runner" to the new one called "nrunner".
>
> Among the changes from one implementation to the other, is the fact
> that "nrunner" will run tests in parallel by default.  This is *not
> yet* enabled by default on "make check-acceptance", but users can
> choose to use simply by setting the "AVOCADO_RUNNER" variable, that
> is:
>
>    make AVOCADO_RUNNER=nrunner check-acceptance
>
> If you are curious about the architectural differences of the nrunner,
> please refer to:
>
>    
> https://avocado-framework.readthedocs.io/en/91.0/guides/contributor/chapters/runners.html
>
> One other noteworthy proposal is a convention to tag tests that either
> have known issues, or that touch on QEMU features that have known
> issues.  By tagging those tests accordingly, they will be
> automatically excluded from the regular execution with "make
> check-acceptance".
>
> Finally, some updates to assets locations and some tests refactors and
> cleanups.
>
> Cleber Rosa (16):
>    Acceptance Tests: bump Avocado requirement to 91.0
>    Acceptance Tests: improve check-acceptance description
>    Acceptance Tests: add mechanism for listing tests
>    Acceptance Tests: keep track and disable tests with known issues
>    Acceptance Tests: add standard clean up at test tearDown()
>    Acceptance Tests: use extract from package from avocado.utils
>    Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert
>    acceptance/tests/vnc.py: use explicit syntax for enabling passwords
>    tests/acceptance/boot_xen.py: merge base classes
>    tests/acceptance/boot_xen.py: unify tags
>    tests/acceptance/boot_xen.py: fetch kernel during test setUp()
>    tests/acceptance/boot_xen.py: removed unused import
>    tests/acceptance/boot_xen.py: use class attribute
>    tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update
>    tests/acceptance/ppc_prep_40p.py: clean up unused import
>    tests/acceptance/ppc_prep_40p.py: unify tags
>
>   docs/devel/testing.rst                    | 40 ++++++++++
>   tests/Makefile.include                    | 15 +++-
>   tests/acceptance/avocado_qemu/__init__.py |  1 +
>   tests/acceptance/boot_linux_console.py    | 93 +++++++++--------------
>   tests/acceptance/boot_xen.py              | 54 ++++---------
>   tests/acceptance/machine_rx_gdbsim.py     |  3 +
>   tests/acceptance/ppc_prep_40p.py          | 17 ++---
>   tests/acceptance/replay_kernel.py         | 18 ++---
>   tests/acceptance/tcg_plugins.py           |  2 +-
>   tests/acceptance/vnc.py                   |  2 +-
>   tests/requirements.txt                    |  2 +-
>   11 files changed, 128 insertions(+), 119 deletions(-)