Re: [PATCH v5 00/11] virtio-mem: vfio support

[Alex Williamson]

On Tue, 16 Feb 2021 19:49:09 +0100
David Hildenbrand <david@redhat.com> wrote:

> On 16.02.21 19:33, Alex Williamson wrote:
> > On Mon, 15 Feb 2021 15:03:43 +0100
> > David Hildenbrand <david@redhat.com> wrote:
> >   
> >> On 08.02.21 09:28, David Hildenbrand wrote:  
> >>> On 27.01.21 13:45, Michael S. Tsirkin wrote:  
> >>>> On Thu, Jan 21, 2021 at 12:05:29PM +0100, David Hildenbrand wrote:  
> >>>>> A virtio-mem device manages a memory region in guest physical address
> >>>>> space, represented as a single (currently large) memory region in QEMU,
> >>>>> mapped into system memory address space. Before the guest is allowed to 
> >>>>> use
> >>>>> memory blocks, it must coordinate with the hypervisor (plug blocks). 
> >>>>> After
> >>>>> a reboot, all memory is usually unplugged - when the guest comes up, it
> >>>>> detects the virtio-mem device and selects memory blocks to plug (based 
> >>>>> on
> >>>>> resize requests from the hypervisor).
> >>>>>
> >>>>> Memory hot(un)plug consists of (un)plugging memory blocks via a 
> >>>>> virtio-mem
> >>>>> device (triggered by the guest). When unplugging blocks, we discard the
> >>>>> memory - similar to memory balloon inflation. In contrast to memory
> >>>>> ballooning, we always know which memory blocks a guest may actually use 
> >>>>> -
> >>>>> especially during a reboot, after a crash, or after kexec (and during
> >>>>> hibernation as well). Guests agreed to not access unplugged memory 
> >>>>> again,
> >>>>> especially not via DMA.
> >>>>>
> >>>>> The issue with vfio is, that it cannot deal with random discards - for 
> >>>>> this
> >>>>> reason, virtio-mem and vfio can currently only run mutually exclusive.
> >>>>> Especially, vfio would currently map the whole memory region (with 
> >>>>> possible
> >>>>> only little/no plugged blocks), resulting in all pages getting pinned 
> >>>>> and
> >>>>> therefore resulting in a higher memory consumption than expected 
> >>>>> (turning
> >>>>> virtio-mem basically useless in these environments).
> >>>>>
> >>>>> To make vfio work nicely with virtio-mem, we have to map only the 
> >>>>> plugged
> >>>>> blocks, and map/unmap properly when plugging/unplugging blocks 
> >>>>> (including
> >>>>> discarding of RAM when unplugging). We achieve that by using a new 
> >>>>> notifier
> >>>>> mechanism that communicates changes.  
> >>>>
> >>>> series
> >>>>
> >>>> Acked-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>
> >>>> virtio bits
> >>>>
> >>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>
> >>>> This needs to go through vfio tree I assume.  
> >>>
> >>> Thanks Michael.
> >>>
> >>> @Alex, what are your suggestions?  
> >>
> >> Gentle ping.  
> > 
> > Sorry for the delay.  It looks to me like patches 1, 8, and 9 are
> > Memory API that are still missing an Ack from Paolo.  I'll toss in my
> > A-b+R-b for patches 6 and 7.  I don't see that this necessarily needs
> > to go in through vfio, I'm more than happy if someone else wants to
> > grab it.  Thanks,  
> 
> Thanks, I assume patch #11 is fine with you as well?

Yep, sent my acks for it as well.  Thanks,

Alex
 
> @Paolo, it would be great if I can get your feedback on patch #1. I have 
> more stuff coming up that will reuse RamDiscardMgr (i.e., for better 
> migration handling and better guest memory dump handling).
>