[Qemu-devel] [RFC v3 00/10] KVM platform device passthrough

[Eric Auger]

This RFC series aims at enabling KVM platform device passthrough.
It implements a VFIO platform device and offers the capability to
instantiate this VFIO device in mach-virt.

The VFIO platform device uses an host VFIO platform driver which must
be bound to the assigned device prior to the QEMU system start.

- the guest can directly access the device register space
- assigned device IRQs are transparently routed to the guest by
  QEMU/KVM (2 methods currently are supported)
- iommu is transparently programmed to prevent the device from
  accessing physical pages outside of the guest address space

The patch series is made of the following patch files

1) provides a mach_virt implementation where a VFIO device is
   instantiated at a fixed location

2) addresses A.Williamson's comment to have the platform device code
   separated from the PCI device code. The VFIO device is moved in a
   new directory hw/vfio/

3) provides a VFIO platform device that supports MMIO direct accesses.
   the vfio device was reworked to factorize at best the code between
   the PCI device and the platform device.

4) simplifies pci device trace calls using common "name" field

5) provides initial IRQ support. The device IRQ now are routed to the
   guest. IRQ handling is based on eventfds handled on user side.
   End of interrupt is detected by trapping guest access to MMIO.
   Functional but suffers from some performance limitations.

6) enables the QEMU end-user to dynamically assign the device
   from command line, using -device option. Request a minimal
   knowledge from the end-user (vfio driver name and compatibility).
   From that point on the VFIO platform device becomes fully generic.
   a single compat string is supported. A single MMIO region is
   supported.

7) regions are IOMMU mapped as executable. This feature is requested
   for some DMA devices that fetch code from some regions (typically
   the PL330).

8) Add support for multi compat strings. This feature is requested
   for Primecell devices

9) forces eventfd notifying mechanism

10) Introduces a new way of IRQ routing (based on KVM irqfd/GSI
   routing). This method is far more performant than the one
   introduced in 4) since eventfds are handled on host kernel side
   and also interrupt completion is trapped at GIC level.

v1 (Kim Phillips):
Initial versions for 1, 2, 3

v1->v2 changes (Kim Phillips, Eric Auger):
- reworked split between PCI and platform (3)
- IRQ initial support (5)
- dynamic instantiation (6)

v2->v3 changes (Alvise Rigo, Eric Auger):
- Following Alex recommandations, further efforts to factorize the
  code between PCI, platform usage of VFIOPlatform and VFIORegion
  as base classes (3, 4)
- cleanup following Kim's comments
- multiple IRQ support mechanics should be in place although not
  tested
- Better handling of MMIO multiple regions
- New features and fixes by Alvise (7, 8, 9)
- irqfd support (10)

This patch has the following dependencies on kernel side:

- [RFC Patch v5 0/11] VFIO support for platform devices
http://www.spinics.net/lists/kvm/msg102309.html
- [Patch] ARM: KVM: Handle IPA unmapping on memory region deletion
https://patches.linaro.org/27691/
- [PATCH v2] ARM: KVM: add irqfd and irq routing support
https://patches.linaro.org/29896/
- [PATCH] ARM: KVM: Enable the KVM-VFIO device
https://lists.cs.columbia.edu/pipermail/kvmarm/2014-March/008629.html
- [PATCH] ARM: KVM: user_mem_abort: support stage 2 MMIO page mapping
https://lists.cs.columbia.edu/pipermail/kvmarm/2014-March/008630.html

The patch series was tested on Calxeda Midway (ARMv7) where one xgmac
is assigned to KVM host while the second one is assigned to the guest.

Tentative Plan:
- further IRQ handling optimizations (removal of maintenance IRQ)
- unbind/migration/reset problematics
- multi-instantiation testing
- multiple IRQ testing
- management of platform devices with more complex device tree node

Here are the instructions to test on a Calxeda Midway:

https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway

git://git.linaro.org/people/eric.auger/linux.git (branch irqfd_integ_v2)
git://git.linaro.org/people/eric.auger/qemu.git (branch vfio-dev-integ-RFCv3)

Best Regards

Eric


Alvise Rigo (3):
  Add EXEC_FLAG to VFIO DMA mappings
  Add AMBA devices support to VFIO
  Always use eventfd as notifying mechanism

Eric Auger (4):
  vfio: simplifed DPRINTF calls using device name
  vfio: Add initial IRQ support in platform device
  virt: Assign a VFIO platform device with -device option
  vfio: Add irqfd support in platform device

Kim Phillips (3):
  hw/arm/virt: add a xgmac device
  vfio: move hw/misc/vfio.c to hw/vfio/pci.c
  vfio: add vfio-platform support

 LICENSE                        |    2 +-
 MAINTAINERS                    |    2 +-
 hw/Makefile.objs               |    1 +
 hw/arm/virt.c                  |  238 +++++-
 hw/intc/arm_gic_kvm.c          |    1 +
 hw/misc/Makefile.objs          |    1 -
 hw/vfio/Makefile.objs          |    5 +
 hw/vfio/common.c               |  854 ++++++++++++++++++++++
 hw/{misc/vfio.c => vfio/pci.c} | 1562 ++++++++++------------------------------
 hw/vfio/platform.c             |  733 +++++++++++++++++++
 hw/vfio/vfio-common.h          |  153 ++++
 linux-headers/linux/vfio.h     |    3 +
 12 files changed, 2378 insertions(+), 1177 deletions(-)
 create mode 100644 hw/vfio/Makefile.objs
 create mode 100644 hw/vfio/common.c
 rename hw/{misc/vfio.c => vfio/pci.c} (65%)
 create mode 100644 hw/vfio/platform.c
 create mode 100644 hw/vfio/vfio-common.h

-- 
1.8.3.2