Re: [PATCH 2/2] tests/avocado/boot_linux.py: re-enable test-case for ppc64

[Kautuk Consul]

On 2023-03-27 17:07:30, Alex Bennée wrote:
> 
> Kautuk Consul <kconsul@linux.vnet.ibm.com> writes:
> 
> > Fixes c0c8687ef0("tests/avocado: disable BootLinuxPPC64 test in CI").
> >
> > Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the test-case
> > for PPC64. On investigation, this turns out to be an issue with the
> > time taken for downloading the Fedora 31 qcow2 image being included
> > within the test-case timeout.
> > Re-enable this test-case by setting the timeout to 360 seconds just
> > before launching the downloaded VM image.
> >
> > Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
> > Reported-by: Alex Bennée <alex.bennee@linaro.org>
> > Tested-by: Hariharan T S hariharan.ts@linux.vnet.ibm.com
> 
> It doesn't really address the principle problem that the
> boot_linux.py:BootLinuxPPC64.test_pseries_tcg is super heavyweight for
> only 2% extra coverage of the executed lines.
By re-enabling this test-case we will ensure that PPC64 part of qemu
works okay in terms of basic linux boot. Without this we will have
a regression in the sense that there won't be any way to test out
basic linux boot for PPC64.
> 
> What we really need is a script so we can compare the output between the
> two jsons:
> 
>   gcovr --json --exclude-unreachable-branches --print-summary -o 
> coverage.json --root ../../ . *.p
> 
> because I suspect we could make up that missing few % noodling the
> baseline test a bit more.
Can you tell me how you check code coverage with and without this
test-case ? I am kind of new to qemu so it would be nice to know how you
do this. And I am trying to increase the code coverage by improving
the baseline test by including more devices in the qemu-system-ppc64
command line so I would appreciate any tips on how to do that also.
> 
> > ---
> >  tests/avocado/boot_linux.py | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
> > index be30dcbd58..c3869a987c 100644
> > --- a/tests/avocado/boot_linux.py
> > +++ b/tests/avocado/boot_linux.py
> > @@ -91,9 +91,9 @@ class BootLinuxPPC64(LinuxTest):
> >      :avocado: tags=arch:ppc64
> >      """
> >  
> > +    # timeout for downloading new VM image.
> >      timeout = 360
> >  
> > -    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
> >      def test_pseries_tcg(self):
> >          """
> >          :avocado: tags=machine:pseries
> > @@ -101,6 +101,10 @@ def test_pseries_tcg(self):
> >          """
> >          self.require_accelerator("tcg")
> >          self.vm.add_args("-accel", "tcg")
> > +
> > +        # timeout for actual Linux PPC boot test
> > +        self.timeout = 360
> > +
> >          self.launch_and_wait(set_up_ssh_connection=False)
> 
> 
> -- 
> Alex Bennée
> Virtualisation Tech Lead @ Linaro
>