[RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream

[Philippe Mathieu-Daudé]

TL;DR: Patch 13 is an experiment to dynamically generate pipelines.

Hi,

This series is an intent to allow git forks to use different set
of jobs than the ones used by the mainstream repository.

Currently, a fork gets the mainstream CI pipeline YAML configuration
and runs its set of jobs by default. Mainstream tests many cases
which might not be always useful to forks. Beside, one mainstream
pipeline might be a waste of ressources when forks provide their
own runners to their gitlab namespace, or use private instances.

This is not a new / unique problem to our community, and Gitlab
already figured it out. They describe this case as Upstream (our
project mainline) / Downstream (a fork).

The pipeline is divided in 2 steps. First the "upstream" part, which
does some evaluation, and depending on the result, triggers the
"downstream" part. The "upstream" is committed into mainline, and
dynamically generate the "downstream" pipeline. This allows forks
to easily rebase with mainstream, avoiding merge conflicts.

This might not be adapted to the QEMU use case, but I wanted to give
it a try.

Gitlab recommends to use job templates. To be able to re-use /
include templates, they can not use YAML anchors. All current jobs
has first to be converted to use the 'extends' keyworkd.
This is done by patches 1-5 (already reviewed).

Patches 6-12 extract various generic jobs (which can be reused by
forks) as template.

Patches 13-14 follow Gitlab guideline to generate dynamic pipelines

Patch 15 is an example of how a fork could use mainstream templates
to maintain his own set of jobs.

The PRO is forks can contribute to templates.

Regards,

Phil.

Philippe Mathieu-Daudé (15):
  gitlab-ci: Replace YAML anchors by extends (container_job)
  gitlab-ci: Replace YAML anchors by extends (native_build_job)
  gitlab-ci: Replace YAML anchors by extends (native_test_job)
  gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
  gitlab-ci: Rename acceptance_test_job -> integration_test_job
  gitlab-ci: Extract container job template to container-template.yml
  gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml
  gitlab-ci: Extract DCO/style check jobs to checks.yml
  gitlab-ci: Extract build stages to stages.yml
  gitlab-ci: Extract all default build/test jobs to buildtest.yml
  gitlab-ci: Extract core container jobs to container-core.yml
  gitlab-ci: Move current job set to qemu-project.yml
  gitlab-ci: Switch to dynamically generated pipelines
  gitlab-ci: Allow forks to use different set of jobs
  gitlab-ci: Use my own set of jobs for CI pipeline

 .gitlab-ci.d/buildtest-template.yml  |  80 +++
 .gitlab-ci.d/buildtest.yml           | 744 +++++++++++++++++++++++
 .gitlab-ci.d/checks.yml              |  24 +
 .gitlab-ci.d/container-core.yml      |  17 +
 .gitlab-ci.d/container-template.yml  |  22 +
 .gitlab-ci.d/containers.yml          | 108 ++--
 .gitlab-ci.d/crossbuild-template.yml |  41 ++
 .gitlab-ci.d/crossbuilds.yml         |  43 +-
 .gitlab-ci.d/philmd.yml              |  33 ++
 .gitlab-ci.d/qemu-project.yml        |   8 +
 .gitlab-ci.d/stages.yml              |   8 +
 .gitlab-ci.yml                       | 845 +--------------------------
 12 files changed, 1033 insertions(+), 940 deletions(-)
 create mode 100644 .gitlab-ci.d/buildtest-template.yml
 create mode 100644 .gitlab-ci.d/buildtest.yml
 create mode 100644 .gitlab-ci.d/checks.yml
 create mode 100644 .gitlab-ci.d/container-core.yml
 create mode 100644 .gitlab-ci.d/container-template.yml
 create mode 100644 .gitlab-ci.d/crossbuild-template.yml
 create mode 100644 .gitlab-ci.d/philmd.yml
 create mode 100644 .gitlab-ci.d/qemu-project.yml
 create mode 100644 .gitlab-ci.d/stages.yml

-- 
2.26.3