Re: changing tests/qtest/meson.build causes unnecessary rebuilding

[Thomas Huth]

On 21/01/2021 14.12, Paolo Bonzini wrote:
> On 21/01/21 12:56, Peter Maydell wrote:
> > $ make -C build/arm-clang/ -j8
> > make: Entering directory
> > '/home/petmay01/linaro/qemu-from-laptop/qemu/build/arm-clang'
> > [1/23] Generating qemu-version.h with a meson_exe.py custom command
> > make: Leaving directory
> > '/home/petmay01/linaro/qemu-from-laptop/qemu/build/arm-clang'
> > $ touch tests/qtest/meson.build
> > $ make -C build/arm-clang/ -j8
> > make: Entering directory
> > '/home/petmay01/linaro/qemu-from-laptop/qemu/build/arm-clang'
> > /usr/bin/ninja  build.ninja && touch build.ninja.stamp
> > [0/1] Regenerating build files.
> > The Meson build system
> > [...]
> >
> > It then goes on to rebuild hundreds or thousands of files, most of
> > which are not even in tests/. (Oddly, the exact set of files recompiled
> > seems to vary from run to run.)
> >
> > Forcing a full rebuild of all of QEMU seems rather excessive when
> > the change was likely "add a new test case"...
>
> This is "avoid build.ninja changes due to order of hash table iteration" 
> (https://github.com/mesonbuild/meson/pull/7900/).  I think Meson 0.57 (with 
> the fix) should be out soon, hopefully before 6.0.
>
> Alternatively you can try to bug your distro to include the patches, they 
> are pretty safe.

Not sure if it is related, but I noticed that we are also rebuilding a lot 
of files in the gitlab-CI that we did not before the meson conversion, 
especially in the check-system-* jobs, e.g:

https://gitlab.com/qemu-project/qemu/-/jobs/977344949#L366

The check-system-* jobs should normally take the artifacts from the 
build-system-* jobs and thus hardly recompile anything at all.

A part of the problem seems to be that we check out the submodules again, I 
can get rid of the superfluous reconfiguration step by adding something like:

diff -u a/.gitlab-ci.yml b/.gitlab-ci.yml
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -39,6 +39,8 @@ include:
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
     - cd build
+    - touch *
+    - make git-submodule-update
     - find . -type f -exec touch {} +
     - make $MAKE_CHECK_ARGS

... but still, the jobs then recompile almost all files afterwards... could 
that be related to that meson problem, too?

 Thomas