[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: EBUSY when using NVMe Block Driver with multiple devices in the same
From: |
Stefan Hajnoczi |
Subject: |
Re: EBUSY when using NVMe Block Driver with multiple devices in the same IOMMU group |
Date: |
Wed, 24 Aug 2022 13:18:59 -0400 |
On Tue, Aug 23, 2022 at 10:36:00PM +0000, Martin Oliveira wrote:
> Hello,
>
> I'm trying to use the QEMU NVMe userspace driver and I'm hitting an error
> when trying to use more than one device from an IOMMU group:
>
> Failed to open VFIO group file: /dev/vfio/39: Device or resource busy
>
> If devices belong to different IOMMU groups, then it works as expected.
>
> For each device, I bind it to vfio-pci and then use something like this:
>
> -drive file=nvme://0000:26:00.0,if=none,id=drive0,format=raw
> -device virtio-blk,drive=drive0,id=virtio0,serial=nvme0
>
> Using the file-based protocol (file=/dev/nvme0n1) works with multiple devices
> from the same group.
>
> My host is running a 5.19 kernel and QEMU is the latest upstream
> (a8cc5842b5cb).
First, multiple QEMU instances cannot access nvme:// devices sharing the
same IOMMU group. I don't think this will ever be possible because it
opens a backdoor around process memory isolation.
However, a single QEMU (or qemu-storage-daemon) instance should be able
to access multiple nvme:// devices in the same IOMMU group.
Unfortunately the code currently doesn't support that.
util/vfio-helpers.c:qemu_vfio_init_pci() has no logic for sharing
groups/containers. Opening the group fails with EBUSY because the kernel
only allows the file to be opened once at any given time.
It's possible to extend the util/vfio-helpers.c code to reuse VFIO
groups (and share VFIO containers), but I'm not aware of anyone who is
currently working on that.
Stefan
signature.asc
Description: PGP signature