Re: [PATCH v2 0/7] tests/acceptance: Handle tests with "cpu" tag

[Cleber Rosa]

On Thu, Apr 08, 2021 at 04:52:30PM -0300, Wainer dos Santos Moschetta wrote:
> Currently the acceptance tests tagged with "machine" have the "-M TYPE"
> automatically added to the list of arguments of the QEMUMachine object.
> In other words, that option is passed to the launched QEMU. On this
> series it is implemented the same feature but instead for tests marked
> with "cpu".
> 
> There is a caveat, however, in case the test needs additional arguments to
> the CPU type they cannot be passed via tag, because the tags parser split
> values by comma. For example, in tests/acceptance/x86_cpu_model_versions.py,
> there are cases where:

Hi Wainer,

I've created an Avocado issue to hopefully get rid of this limitation:

   https://github.com/avocado-framework/avocado/issues/4541

> 
>   * -cpu is set to 
> "Cascadelake-Server,x-force-features=on,check=off,enforce=off"
>   * if it was tagged like 
> "cpu:Cascadelake-Server,x-force-features=on,check=off,enforce=off"
>     then the parser would break it into 4 tags ("cpu:Cascadelake-Server",
>     "x-force-features=on", "check=off", "enforce=off")
>   * resulting on "-cpu Cascadelake-Server" and the remaining arguments are 
> ignored.
> 
> It was introduced the avocado_qemu.Test.set_vm_arg() method to deal with
> cases like the example above, so that one can tag it as 
> "cpu:Cascadelake-Server"
> AND call self.set_vm_args('-cpu', 
> "Cascadelake-Server,x-force-features=on,check=off,enforce=off"),
> and that results on the reset of the initial value of -cpu.
>

So for now this seems reasonable enough.

> This series was tested on CI 
> (https://gitlab.com/wainersm/qemu/-/pipelines/277376246)
> and with the following code:
> 
> from avocado_qemu import Test
> 
> class CPUTest(Test):
>     def test_cpu(self):
>         """
>         :avocado: tags=cpu:host
>         """
>         # The cpu property is set to the tag value, or None on its absence
>         self.assertEqual(self.cpu, "host")
>         # The created VM has the '-cpu host' option
>         self.assertIn("-cpu host", " ".join(self.vm._args))
>         self.vm.launch()
> 
>     def test_cpu_none(self):
>         self.assertEqual(self.cpu, None)
>         self.assertNotIn('-cpu', self.vm._args)
> 
>     def test_cpu_reset(self):
>         """
>         :avocado: tags=cpu:host
>         """
>         self.assertIn("-cpu host", " ".join(self.vm._args))
>         self.set_vm_arg("-cpu", "Cascadelake-Server,x-force-features=on")
>         self.assertNotIn("-cpu host", " ".join(self.vm._args))
>         self.assertIn("-cpu Cascadelake-Server,x-force-features=on", " 
> ".join(self.vm._args))
>

We should not let this type of testing go to waste, so it's about time
to set aside a directory for tests that are about the framework,
rather than end user functionality.  I'll take a look at that.

Cheers,
- Cleber.

signature.asc
Description: PGP signature