Re: [PATCH v3 00/13] vfio/migration: Device dirty page tracking

[Alex Williamson]

On Sun, 5 Mar 2023 23:33:35 +0000
Joao Martins <joao.m.martins@oracle.com> wrote:

> On 05/03/2023 20:57, Alex Williamson wrote:
> > On Sat,  4 Mar 2023 01:43:30 +0000
> > Joao Martins <joao.m.martins@oracle.com> wrote:
> >   
> >> Hey,
> >>
> >> Presented herewith a series based on the basic VFIO migration protocol v2
> >> implementation [1].
> >>
> >> It is split from its parent series[5] to solely focus on device dirty
> >> page tracking. Device dirty page tracking allows the VFIO device to
> >> record its DMAs and report them back when needed. This is part of VFIO
> >> migration and is used during pre-copy phase of migration to track the
> >> RAM pages that the device has written to and mark those pages dirty, so
> >> they can later be re-sent to target.
> >>
> >> Device dirty page tracking uses the DMA logging uAPI to discover device
> >> capabilities, to start and stop tracking, and to get dirty page bitmap
> >> report. Extra details and uAPI definition can be found here [3].
> >>
> >> Device dirty page tracking operates in VFIOContainer scope. I.e., When
> >> dirty tracking is started, stopped or dirty page report is queried, all
> >> devices within a VFIOContainer are iterated and for each of them device
> >> dirty page tracking is started, stopped or dirty page report is queried,
> >> respectively.
> >>
> >> Device dirty page tracking is used only if all devices within a
> >> VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is
> >> used, and if that is not supported as well, memory is perpetually marked
> >> dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW
> >> support, the last two usually have the same effect of perpetually
> >> marking all pages dirty.
> >>
> >> Normally, when asked to start dirty tracking, all the currently DMA
> >> mapped ranges are tracked by device dirty page tracking. If using a
> >> vIOMMU we block live migration. It's temporary and a separate series is
> >> going to add support for it. Thus this series focus on getting the
> >> ground work first.
> >>
> >> The series is organized as follows:
> >>
> >> - Patches 1-7: Fix bugs and do some preparatory work required prior to
> >>   adding device dirty page tracking.
> >> - Patches 8-10: Implement device dirty page tracking.
> >> - Patch 11: Blocks live migration with vIOMMU.
> >> - Patches 12-13 enable device dirty page tracking and document it.
> >>
> >> Comments, improvements as usual appreciated.  
> > 
> > Still some CI failures:
> > 
> > https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474
> > 
> > The docker failures are normal, afaict the rest are not.  Thanks,
> >   
> 
> Ugh, sorry
> 
> The patch below scissors mark (and also attached as a file) fixes those build
> issues. I managed to reproduce on i386 target builds, and these changes fix my
> 32-bit build.
> 
> I don't have a working Gitlab setup[*] though to trigger the CI to enable to
> wealth of targets it build-tests. If you could kindly test the patch attached 
> in
> a new pipeline (applied on top of the branch you just build) below to 
> understand
> if the CI gets happy. I will include these changes in the right patches 
> (patch 8
> and 10) for the v4 spin.

Looks like this passes:

https://gitlab.com/alex.williamson/qemu/-/pipelines/796750136

Thanks,
Alex