Re: [PATCH] hostmem: default the amount of prealloc-threads to smp-cpus

[Daniel P . Berrangé]

On Wed, May 18, 2022 at 03:02:48PM +0200, Dario Faggioli wrote:
> On Wed, 2022-05-18 at 12:17 +0200, Igor Mammedov wrote:
> > On Tue, 17 May 2022 20:46:50 +0200
> > Paolo Bonzini <pbonzini@redhat.com> wrote:
> > > > diff --git a/backends/hostmem.c b/backends/hostmem.c
> > > > index a7bae3d713..624bb7ecd3 100644
> > > > --- a/backends/hostmem.c
> > > > +++ b/backends/hostmem.c
> > > > @@ -274,7 +274,7 @@ static void host_memory_backend_init(Object
> > > > *obj)
> > > >       backend->merge = machine_mem_merge(machine);
> > > >       backend->dump = machine_dump_guest_core(machine);
> > > >       backend->reserve = true;
> > > > -    backend->prealloc_threads = 1;
> > > > +    backend->prealloc_threads = machine->smp.cpus;
> > > >   }
> > > >   
> > > >   static void host_memory_backend_post_init(Object *obj)  
> > > 
> > > Queued, thanks.
> > 
> > PS:
> > There is no good default in this case (whatever number is picked
> > it could be good or bad depending on usecase).
> > 
> That is fair enough. What we observed, however, is that, with QEMU 5.2,
> starting a 1024G VM takes ~34s.
> 
> Then you just update QEMU to > 5.2 (and don't do/changing anything
> else) and the same VM now takes ~4m30s to start.
> 
> If users are managing QEMU via Libvirt *and* have _at_least_ Libvirt
> 8.2, they can indeed set, e.g., <allocation mode='immediate'
> threads='NNN'/> (provided they can understand where the problem is, and
> figure out that this is the solution).

I think you get the QEMU version numbers a bit mixed up based on
what i see in git history

Originally mem prellocation was single threaded and slow.

In v2.8.1 it switched to multi threaded with nthreads==vcpus

  commit 1e356fc14beaa3ece6c0e961bd479af58be3198b
  Author: Jitendra Kolhe <jitendra.kolhe@hpe.com>
  Date:   Fri Feb 24 09:01:43 2017 +0530

    mem-prealloc: reduce large guest start-up and migration time.

This applied to --mem-prealloc and --object memory-backend*,prealloc=on


In v5.0.0 the prealloc-threads property was introduced with

  commit ffac16fab33bb42f17e47624985220c1fd864e9d
  Author: Igor Mammedov <imammedo@redhat.com>
  Date:   Wed Feb 19 11:09:50 2020 -0500

    hostmem: introduce "prealloc-threads" property

This changed it so that --mem-prealloc stil uses nthreads=vcpus
but --object memory-backend,prealloc=on regressed to nthreads=1

When picking defaults there is never a perfect answer, it
is more a matter of the least-worst option.

It is pretty clear that nthreads=1 is terrible for any
large VMs. Defaulting it to nvcpus made conceptual sense
as the user has implicit said that they expect the VM to
be able to consume nvcpus worth of CPU time on the host,
so we might as well consume that allotted resource.

I struggle to come up with a compelling reason why it is
better to only use 1 single thread for preallocation. There
might be some niches where its useful but I can't see it
being the common case desirable behaviour.

Having different defaults based on how you configure it
is also especially unplesant experience.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|