Re: [PATCH] meson: adjust timeouts for some slower tests

[Daniel P . Berrangé]

On Thu, Feb 11, 2021 at 12:30:35PM +0100, Paolo Bonzini wrote:
> On 09/02/21 18:58, Daniel P. Berrangé wrote:
> > On Tue, Feb 09, 2021 at 06:45:41PM +0100, Paolo Bonzini wrote:
> > > Adjust the timeouts for the longest running tests.  These are the
> > > times that I measured and the corresponding timeouts.  For generic
> > > qtests, the target that reported the longest runtime is included.
> > > 
> > > unit tests:
> > >      test-crypto-tlscredsx509        13.15s   60s
> > >      test-crypto-tlssession          14.12s   60s
> > 
> > The default meson timeout is 30 seconds which is enough for these
> > tests. Of course larger timeouts give more headroom.
> 
> This was a relatively fast run, I've had them take as little as 7s and as
> much as 25s on the same machine.  I suspect it's because the machine has a
> very slow NFS home directory (yes those things still exist :)). In general a
> 2x-ish headroom makes sense in case someone is doing a build at the same
> time as a test run.
> 
> By the way, with Meson 0.57 there's the possibility of specifying "infinite
> timeout", and this could be used for the benchmarks.  Giving slower tests a
> higher priority is also a good idea, and even though this is not guaranteed
> in theory, Make ends up taking into account the priority as well.  With
> these tweaks "meson test" and "make check" (minus check-block of course)
> both clock at 2:20s, which is exactly the time it takes to run the
> longest-running test.
> 
> I will also give "meson test" a shot on the GitLab runners before posting
> v2, and see if it needs a timeout multiplier.

I'd say we should set the default timeouts so that they are suitable for
our automated GitLab CI, and what we expect typical developer platforms.

Timeout multiplier should only be needed if users are running on outdated
hardware, or under slow emulation.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|