Re: [PULL 00/31] various fixes (testing, plugins, gitdm)

[Philippe Mathieu-Daudé]

On 20/3/23 15:15, Daniel P. Berrangé wrote:
> On Mon, Mar 20, 2023 at 01:42:46PM +0000, Peter Maydell wrote:
> > On Sat, 18 Mar 2023 at 11:46, Alex Bennée <alex.bennee@linaro.org> wrote:
> > > The following changes since commit 74c581b6452394e591f13beba9fea2ec0688e2f5:
> > >
> > >    Merge tag 'trivial-branch-for-8.0-pull-request' of 
> > > https://gitlab.com/laurent_vivier/qemu into staging (2023-03-17 14:22:01 +0000)
> > >
> > > are available in the Git repository at:
> > >
> > >    https://gitlab.com/stsquad/qemu.git tags/pull-for-8.0-170323-4
> > >
> > > for you to fetch changes up to 4f2c431acd43d0aa505494229d05fa343762f272:
> > >
> > >    qtests: avoid printing comments before g_test_init() (2023-03-17
> > >    17:50:19 +0000)
> > >
> > > You can see my CI run on the branch here:
> > >
> > >    https://gitlab.com/stsquad/qemu/-/pipelines/810271620
> > >
> > > The failures:
> > >
> > >    FreeBSD's time out on a migration test
> > >    Centos8 Stream because my private runner needs more disk space
> > >
> > > ----------------------------------------------------------------
> > > Misc fixes for 8.0 (testing, plugins, gitdm)
> > >
> > >    - update Alpine image used for testing images
> > >    - include libslirp in custom runner build env
> > >    - update gitlab-runner recipe for CentOS
> > >    - update docker calls for better caching behaviour
> > >    - document some plugin callbacks
> > >    - don't use tags to define drives for lkft baseline tests
> > >    - fix missing clear of plugin_mem_cbs
> > >    - fix iotests to report individual results
> > >    - update the gitdm metadata for contributors
> > >    - avoid printing comments before g_test_init()
> >
> > This seems to consistently fail an avocado test on the
> > centos-stream-8-x86_64 job:
> > (21/51) tests/avocado/multiprocess.py:Multiprocess.test_multiprocess_x86_64:
> > ERROR: ConnectError: Failed to establish session: EOFError\n Exit
> > code: 1\n Command: ./qemu-system-x86_64 -display none -vga none
> > -chardev socket,id=mon,fd=17 -mon chardev=mon,mode=control -machine
> > x-remote -nodefaults -device lsi53c895a,id=lsi1 -object x-remote-o...
> > (0.10 s)
> >
> > https://gitlab.com/qemu-project/qemu/-/jobs/3962028269
> > https://gitlab.com/qemu-project/qemu/-/jobs/3965134190
> >
> >
> >
> > The iotests also don't seem to pass on the OpenBSD VM after this;
> > which test fails varies from run to run but the common factor
> > is a complaint about running out of disk space:
>
> This must be caused by the change in the way we register the
> iotests with meson, as I don't see any other interesting changes
> in this series.

See "05/31 gitlab: update centos-8-stream job", now we call
'make check-avocado' instead of 
scripts/ci/org.centos/stream/8/x86_64/test-avocado.

Per commit d7c2e2b3f4 ("Jobs based on custom runners: add CentOS
Stream 8"):

  The components of this custom job are:

    I) OS build environment setup code:

       - additions to the existing "build-environment.yml" playbook
         that can be used to set up CentOS/EL 8 systems.

       - a CentOS Stream 8 specific "build-environment.yml" playbook
         that adds to the generic one.

   II) QEMU build configuration: a script that will produce binaries
       with features as similar as possible to the ones built and
       packaged on CentOS stream 8.

  III) Scripts that define the minimum amount of testing that the
       binaries built with the given configuration (point II) under the
       given OS build environment (point I) should be subjected to.

Is this host machine (hackbox) installed exactly as per
build-environment.yml?

> At a high level we have four scenarios we're testing
>
>   qcow2 + quick
>   raw  + slow
>   qed + thorough
>   vmdk + thorough
>   vpc + thorough
>
> Previously each scenario was serialized wrt other scenarios, but
> within a scenario things run in parallel. So we could have more
> parallelization than we did in the past.
>
> They were also serialized wrt any other tests QEMU runs.
>
> IIUC, we skip any 'slow' and 'thorough' tests by default
>
>    add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: 
> true)
>    add_test_setup('slow', exclude_suites: ['thorough'], env: ['G_TEST_SLOW=1', 
> 'SPEED=slow'])
>    add_test_setup('thorough', env: ['G_TEST_SLOW=1', 'SPEED=thorough'])
>
>
> so in practice we should only be runing the qcow2 + quick
> tests, not the raw/qed/vmdk/vpc tests.
>
> So the change in parallelism is that we can potentially run the
> block I/O tests in parallel with unit tests. Maybe parallel with
> avocado tests too, I'm not sure ?