Re: [PATCH v5 1/4] Jobs based on custom runners: documentation and configuration placeholder

[Daniel P . Berrangé]

On Tue, Feb 23, 2021 at 11:47:18AM -0500, Cleber Rosa wrote:
> On Tue, Feb 23, 2021 at 05:37:04PM +0100, Philippe Mathieu-Daudé wrote:
> > On 2/23/21 12:25 PM, Thomas Huth wrote:
> > > On 19/02/2021 22.58, Cleber Rosa wrote:
> > >> As described in the included documentation, the "custom runner" jobs
> > >> extend the GitLab CI jobs already in place.  One of their primary
> > >> goals of catching and preventing regressions on a wider number of host
> > >> systems than the ones provided by GitLab's shared runners.
> > >>
> > >> This sets the stage in which other community members can add their own
> > >> machine configuration documentation/scripts, and accompanying job
> > >> definitions.  As a general rule, those newly added contributed jobs
> > >> should run as "non-gating", until their reliability is verified (AKA
> > >> "allow_failure: true").
> > >>
> > >> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > >> ---
> > >>   .gitlab-ci.d/custom-runners.yml | 14 ++++++++++++++
> > >>   .gitlab-ci.yml                  |  1 +
> > >>   docs/devel/ci.rst               | 28 ++++++++++++++++++++++++++++
> > >>   docs/devel/index.rst            |  1 +
> > >>   4 files changed, 44 insertions(+)
> > >>   create mode 100644 .gitlab-ci.d/custom-runners.yml
> > >>   create mode 100644 docs/devel/ci.rst
> > >>
> > >> diff --git a/.gitlab-ci.d/custom-runners.yml
> > >> b/.gitlab-ci.d/custom-runners.yml
> > >> new file mode 100644
> > >> index 0000000000..3004da2bda
> > >> --- /dev/null
> > >> +++ b/.gitlab-ci.d/custom-runners.yml
> > >> @@ -0,0 +1,14 @@
> > >> +# The CI jobs defined here require GitLab runners installed and
> > >> +# registered on machines that match their operating system names,
> > >> +# versions and architectures.  This is in contrast to the other CI
> > >> +# jobs that are intended to run on GitLab's "shared" runners.
> > >> +
> > >> +# Different than the default approach on "shared" runners, based on
> > >> +# containers, the custom runners have no such *requirement*, as those
> > >> +# jobs should be capable of running on operating systems with no
> > >> +# compatible container implementation, or no support from
> > >> +# gitlab-runner.  To avoid problems that gitlab-runner can cause while
> > >> +# reusing the GIT repository, let's enable the recursive submodule
> > >> +# strategy.
> > >> +variables:
> > >> +  GIT_SUBMODULE_STRATEGY: recursive
> > > 
> > > Is it really necessary? I thought our configure script would take care
> > > of the submodules?
> >
> 
> I've done a lot of testing on bare metal systems, and the problems
> that come from reusing the same system and failed cleanups can be very
> frustrating.  It's unfortunate that we need this, but it was the
> simplest and most reliable solution I found.  :/

Hmmm, this makes it sound like the job is not being run in a
fresh pristine checkout. IMHO we need to guarantee that in
general, at which point submodules should "just work", unless
the running is blocking network access ?



Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|