[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 13/21] tests/Makefile: allow control over tags during check-accept
From: |
Philippe Mathieu-Daudé |
Subject: |
[PULL 13/21] tests/Makefile: allow control over tags during check-acceptance |
Date: |
Mon, 27 Sep 2021 19:02:19 +0200 |
From: Willian Rampazzo <willianr@redhat.com>
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.
This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.
This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.
e.g.:
AVOCADO_TAGS="foo bar baz" make check-acceptance
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-2-willianr@redhat.com>
---
docs/devel/testing.rst | 14 ++++++++++++++
tests/Makefile.include | 12 +++++++++---
2 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4a0abbf23d3..d1841e35d57 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -732,6 +732,20 @@ available. On Debian and Ubuntu based systems, depending
on the
specific version, they may be on packages named ``python3-venv`` and
``python3-pip``.
+It is also possible to run tests based on tags using the
+``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
+variable:
+
+.. code::
+
+ make check-acceptance AVOCADO_TAGS=quick
+
+Note that tags separated with commas have an AND behavior, while tags
+separated by spaces have an OR behavior. For more information on Avocado
+tags, see:
+
+
https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
+
The scripts installed inside the virtual environment may be used
without an "activation". For instance, the Avocado test runner
may be invoked by running:
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6e16c05f10b..f6484e5b31d 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
# Any number of command separated loggers are accepted. For more
# information please refer to "avocado --help".
AVOCADO_SHOW=app
-AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+ifndef AVOCADO_TAGS
+ AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
+ $(filter %-softmmu,$(TARGETS)))
+else
+ AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
+endif
$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
$(call quiet-command, \
@@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
get-vm-images
$(call quiet-command, \
$(TESTS_VENV_DIR)/bin/python -m avocado \
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
- --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
- $(AVOCADO_TAGS) \
+ $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
+ --filter-by-tags-include-empty-key) \
+ $(AVOCADO_CMDLINE_TAGS) \
$(if $(GITLAB_CI),,--failfast) tests/acceptance, \
"AVOCADO", "tests/acceptance")
--
2.31.1
- [PULL 03/21] tests/acceptance: add replay kernel test for nios2, (continued)
- [PULL 03/21] tests/acceptance: add replay kernel test for nios2, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 04/21] tests/acceptance: add replay kernel test for alpha, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 06/21] Acceptance tests: add myself as a reviewer for the acceptance tests, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 07/21] Acceptance Tests: add standard clean up at test tearDown(), Philippe Mathieu-Daudé, 2021/09/27
- [PULL 08/21] avocado_qemu: standardize super() call following PEP3135, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 05/21] tests/acceptance: Linux boot test for record/replay, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 09/21] avocado_qemu: fix import module based on isort, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 10/21] avocado_qemu: tweak ssh connect method, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 11/21] avocado_qemu: explicitly return None to avoid R1710, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 12/21] avocado_qemu: fix inheritance order on LinuxTest class, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 13/21] tests/Makefile: allow control over tags during check-acceptance,
Philippe Mathieu-Daudé <=
- [PULL 14/21] docs/devel/testing: add instruction to run a single acceptance test, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 15/21] tests/Makefile: add AVOCADO_TESTS option to make check-acceptance, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 16/21] qemu: Split machine_ppc.py acceptance tests, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 18/21] acceptance/tests/vnc.py: use explicit syntax for enabling passwords, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 17/21] Acceptance Tests: improve check-acceptance description, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 19/21] tests/acceptance/boot_xen.py: removed unused import, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 21/21] tests/acceptance/ppc_prep_40p.py: clean up unused import, Philippe Mathieu-Daudé, 2021/09/27
- [PULL 20/21] tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update, Philippe Mathieu-Daudé, 2021/09/27
- Re: [PULL 00/21] Integration testing patches for 2021-09-27, Philippe Mathieu-Daudé, 2021/09/27