Re: [PATCH v3 00/13] vfio/migration: Device dirty page tracking

[Cédric Le Goater]

On 3/6/23 10:45, Joao Martins wrote:
> On 06/03/2023 02:19, Alex Williamson wrote:
> > On Sun, 5 Mar 2023 23:33:35 +0000
> > Joao Martins <joao.m.martins@oracle.com> wrote:
> >
> > > On 05/03/2023 20:57, Alex Williamson wrote:
> > > > On Sat,  4 Mar 2023 01:43:30 +0000
> > > > Joao Martins <joao.m.martins@oracle.com> wrote:
> > > >    
> > > > > Hey,
> > > > >
> > > > > Presented herewith a series based on the basic VFIO migration protocol v2
> > > > > implementation [1].
> > > > >
> > > > > It is split from its parent series[5] to solely focus on device dirty
> > > > > page tracking. Device dirty page tracking allows the VFIO device to
> > > > > record its DMAs and report them back when needed. This is part of VFIO
> > > > > migration and is used during pre-copy phase of migration to track the
> > > > > RAM pages that the device has written to and mark those pages dirty, so
> > > > > they can later be re-sent to target.
> > > > >
> > > > > Device dirty page tracking uses the DMA logging uAPI to discover device
> > > > > capabilities, to start and stop tracking, and to get dirty page bitmap
> > > > > report. Extra details and uAPI definition can be found here [3].
> > > > >
> > > > > Device dirty page tracking operates in VFIOContainer scope. I.e., When
> > > > > dirty tracking is started, stopped or dirty page report is queried, all
> > > > > devices within a VFIOContainer are iterated and for each of them device
> > > > > dirty page tracking is started, stopped or dirty page report is queried,
> > > > > respectively.
> > > > >
> > > > > Device dirty page tracking is used only if all devices within a
> > > > > VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is
> > > > > used, and if that is not supported as well, memory is perpetually marked
> > > > > dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW
> > > > > support, the last two usually have the same effect of perpetually
> > > > > marking all pages dirty.
> > > > >
> > > > > Normally, when asked to start dirty tracking, all the currently DMA
> > > > > mapped ranges are tracked by device dirty page tracking. If using a
> > > > > vIOMMU we block live migration. It's temporary and a separate series is
> > > > > going to add support for it. Thus this series focus on getting the
> > > > > ground work first.
> > > > >
> > > > > The series is organized as follows:
> > > > >
> > > > > - Patches 1-7: Fix bugs and do some preparatory work required prior to
> > > > >    adding device dirty page tracking.
> > > > > - Patches 8-10: Implement device dirty page tracking.
> > > > > - Patch 11: Blocks live migration with vIOMMU.
> > > > > - Patches 12-13 enable device dirty page tracking and document it.
> > > > >
> > > > > Comments, improvements as usual appreciated.
> > > >
> > > > Still some CI failures:
> > > >
> > > > https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474
> > > >
> > > > The docker failures are normal, afaict the rest are not.  Thanks,
> > > >    
> > >
> > > Ugh, sorry
> > >
> > > The patch below scissors mark (and also attached as a file) fixes those build
> > > issues. I managed to reproduce on i386 target builds, and these changes fix my
> > > 32-bit build.
> > >
> > > I don't have a working Gitlab setup[*] though to trigger the CI to enable to
> > > wealth of targets it build-tests. If you could kindly test the patch attached in
> > > a new pipeline (applied on top of the branch you just build) below to understand
> > > if the CI gets happy. I will include these changes in the right patches (patch 8
> > > and 10) for the v4 spin.
> >
> > Looks like this passes:
> >
> > https://gitlab.com/alex.williamson/qemu/-/pipelines/796750136
> Great, I've staged this fixes in patches 8&10!
>
> I have a sliver of hope that we might still make it by soft freeze (tomorrow?).
> If you think it can still make it, should the rest of the series is good, then I
> can follow up v4 today/tomorrow. Thoughts?

I would say, wait and see if a v4 is needed first. These changes are
relatively easy to fold in.

C.



>         Joao