[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buil
From: |
Thomas Huth |
Subject: |
Re: [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buildtest.yml |
Date: |
Mon, 19 Apr 2021 07:39:48 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 |
On 19/04/2021 01.34, Philippe Mathieu-Daudé wrote:
Extract the build/test jobs run by default on the mainstream
CI into a new file (buildtest.yml) and their templates into
(buildtest-template.yml), to be able to reuse the templates
without having to run all these mainstream jobs by default.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.gitlab-ci.d/buildtest-template.yml | 80 +++
.gitlab-ci.d/buildtest.yml | 744 +++++++++++++++++++++++++
.gitlab-ci.yml | 824 +---------------------------
3 files changed, 825 insertions(+), 823 deletions(-)
create mode 100644 .gitlab-ci.d/buildtest-template.yml
create mode 100644 .gitlab-ci.d/buildtest.yml
I like the idea, the main .gitlab-ci.yml was way to crowded already anyway.
Not sure if "buildtest.yml" is such a good name (since it's not only about
building, but also about testing), but I fail to come up with a better
suggestion...
Just a suggestion for discussion, not sure whether it's such a good idea
yet: Maybe we could split it up even further into "system.yml" (with the
various jobs that only test the "softmmu" binaries), "user.yml" (with the
--disable-system jobs) and "misc.yml" (with mixed jobs) ?
diff --git a/.gitlab-ci.d/buildtest-template.yml
b/.gitlab-ci.d/buildtest-template.yml
new file mode 100644
index 00000000000..b24a153d904
--- /dev/null
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -0,0 +1,80 @@
+.native_build_job_template:
+ stage: build
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ before_script:
+ - JOBS=$(expr $(nproc) + 1)
+ script:
+ - mkdir build
+ - cd build
+ - if test -n "$TARGETS";
+ then
+ ../configure --enable-werror --disable-docs $CONFIGURE_ARGS
--target-list="$TARGETS" ;
+ else
+ ../configure --enable-werror --disable-docs $CONFIGURE_ARGS ;
+ fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
+ - if test -n "$LD_JOBS";
+ then
+ meson configure . -Dbackend_max_links="$LD_JOBS" ;
+ fi || exit 1;
+ - make -j"$JOBS"
+ - if test -n "$MAKE_CHECK_ARGS";
+ then
+ make -j"$JOBS" $MAKE_CHECK_ARGS ;
+ fi
+
+.native_test_job_template:
+ stage: test
+ image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ script:
+ - scripts/git-submodule.sh update
+ $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
+ - cd build
+ - find . -type f -exec touch {} +
+ # Avoid recompiling by hiding ninja with NINJA=":"
+ - make NINJA=":" $MAKE_CHECK_ARGS
+
+.integration_test_job_template:
+ cache:
+ key: "${CI_JOB_NAME}-cache"
+ paths:
+ - ${CI_PROJECT_DIR}/avocado-cache
+ policy: pull-push
+ artifacts:
+ name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+ when: always
+ expire_in: 2 days
+ paths:
+ - build/tests/results/latest/results.xml
+ - build/tests/results/latest/test-results
+ reports:
+ junit: build/tests/results/latest/results.xml
+ before_script:
+ - mkdir -p ~/.config/avocado
+ - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
+ - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
+ >> ~/.config/avocado/avocado.conf
+ - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
+ >> ~/.config/avocado/avocado.conf
+ - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
+ du -chs ${CI_PROJECT_DIR}/avocado-cache ;
+ fi
+ - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+ after_script:
+ - cd build
+ - du -chs ${CI_PROJECT_DIR}/avocado-cache
+
+build-system-alpine:
+ extends: .native_build_job_template
+ needs:
+ - job: amd64-alpine-container
+ variables:
+ IMAGE: alpine
+ TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
+ moxie-softmmu microblazeel-softmmu mips64el-softmmu
+ MAKE_CHECK_ARGS: check-build
+ CONFIGURE_ARGS: --enable-docs --enable-trace-backends=log,simple,syslog
+ artifacts:
+ expire_in: 2 days
+ paths:
+ - .git-submodule-status
+ - build
build-system-alpine should certainly not go into the template file, but into
the file below instead.
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
new file mode 100644
index 00000000000..c24553eab05
--- /dev/null
+++ b/.gitlab-ci.d/buildtest.yml
@@ -0,0 +1,744 @@
[...]
+check-patch:
+ stage: build
+ image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+ needs:
+ job: amd64-centos8-container
+ script: .gitlab-ci.d/check-patch.py
+ except:
+ variables:
+ - $CI_PROJECT_NAMESPACE == 'qemu-project' && $CI_COMMIT_BRANCH ==
'master'
+ variables:
+ GIT_DEPTH: 1000
+ allow_failure: true
+
+check-dco:
+ stage: build
+ image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+ needs:
+ job: amd64-centos8-container
+ script: .gitlab-ci.d/check-dco.py
+ except:
+ variables:
+ - $CI_PROJECT_NAMESPACE == 'qemu-project' && $CI_COMMIT_BRANCH ==
'master'
+ variables:
+ GIT_DEPTH: 1000
These two should not be here anymore since they have been handled in patch
08/15 already.
> +# No targets are built here, just tools, docs, and unit tests. This
> +# also feeds into the eventual documentation deployment steps later
> +build-tools-and-docs-debian:
> + extends: .native_build_job_template
> + needs:
> + job: amd64-debian-container
> + variables:
> + IMAGE: debian-amd64
> + MAKE_CHECK_ARGS: check-unit check-softfloat ctags TAGS cscope
> + CONFIGURE_ARGS: --disable-system --disable-user --enable-docs
--enable-tools
> + artifacts:
> + expire_in: 2 days
> + paths:
> + - build
> +
> +# Prepare for GitLab pages deployment. Anything copied into the
> +# "public" directory will be deployed to $USER.gitlab.io/$PROJECT
> +pages:
> + image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
> + stage: test
> + needs:
> + - job: build-tools-and-docs-debian
> + script:
> + - mkdir -p public
> + # HTML-ised source tree
> + - make gtags
> + - htags -anT --tree-view=filetree -m qemu_init
> + -t "Welcome to the QEMU sourcecode"
> + - mv HTML public/src
> + # Project documentation
> + - make -C build install DESTDIR=$(pwd)/temp-install
> + - mv temp-install/usr/local/share/doc/qemu/* public/
> + artifacts:
> + paths:
> + - public
I think the "pages" job should maybe rather go into a separate file instead
(maybe together with the build-tools-and-docs-debian job)?
Thomas
- [PATCH 04/15] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job), (continued)
- [PATCH 04/15] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job), Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 05/15] gitlab-ci: Rename acceptance_test_job -> integration_test_job, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 06/15] gitlab-ci: Extract container job template to container-template.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 07/15] gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 08/15] gitlab-ci: Extract DCO/style check jobs to checks.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 09/15] gitlab-ci: Extract build stages to stages.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buildtest.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 11/15] gitlab-ci: Extract core container jobs to container-core.yml, Philippe Mathieu-Daudé, 2021/04/18
- [PATCH 12/15] gitlab-ci: Move current job set to qemu-project.yml, Philippe Mathieu-Daudé, 2021/04/18
- [RFC PATCH 13/15] gitlab-ci: Switch to dynamically generated pipelines, Philippe Mathieu-Daudé, 2021/04/18
- [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs, Philippe Mathieu-Daudé, 2021/04/18