Re: [PATCH v3 24/24] tests/avocado: disable BootLinuxPPC64 test in CI

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 28/02/2023 20.06, Alex Bennée wrote:
>> This test is exceptionally heavyweight (nearly 330s) compared to the
>> two (both endians) TuxRun baseline tests which complete in under 160s.
>> The coverage is slightly reduced but a more directed test could make
>> up the difference.
>> tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_ppc64:
>> Overall coverage rate:
>>    lines......: 9.6% (44110 of 458817 lines)
>>    functions..: 16.5% (6767 of 41054 functions)
>>    branches...: 6.0% (13395 of 222634 branches)
>> tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg:
>> Overall coverage rate:
>>    lines......: 11.6% (53408 of 458817 lines)
>>    functions..: 18.7% (7691 of 41054 functions)
>>    branches...: 7.9% (17692 of 224218 branches)
>> So lets skip for GITLAB_CI and also unless AVOCADO_TIMEOUT_EXPECTED
>> is
>> specified by the user.
>
> The explanation sounds somewhat implausible to me.
> AVOCADO_TIMEOUT_EXPECTED should be for jobs where we are not sure
> whether the job really finishes in time, e.g. when compiling QEMU with
> debug flags enabled, and not for jobs that simply run a little bit
> longer (in the latter case, it would be enough to simply bump the
> timeout setting a little bit if necessary). So did you check whether
> you really run into timeout issues here when compiling QEMU with debug
> flags?

Ahh I realise now that I was running into the timeout because it was a
gcov build. I'll drop the AVOACADO_TIMEOUT_EXPECTED bit for now.

>
> Anyway, if you add AVOCADO_TIMEOUT_EXPECTED, then I think you don't
> need GITLAB_CI anymore, since we certainly don't set
> AVOCADO_TIMEOUT_EXPECTED in the gitlab CI.
>
>  Thomas


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro