On 8/3/2023 11:07 PM, Andrew Jones wrote:
On Mon, Jul 31, 2023 at 09:53:17AM +0800, Fei Wu wrote:
riscv virt platform's memory started at 0x80000000 and
straddled the 4GiB boundary. Curiously enough, this choice
of a memory layout will prevent from launching a VM with
a bit more than 2000MiB and PCIe pass-thru on an x86 host, due
to identity mapping requirements for the MSI doorbell on x86,
and these (APIC/IOAPIC) live right below 4GiB.
So just split the RAM range into two portions:
- 1 GiB range from 0x80000000 to 0xc0000000.
- The remainder at 0x100000000
...leaving a hole between the ranges.
Can you elaborate on the use case? Maybe provide details of the host
system and the QEMU command line? I'm wondering why we didn't have
any problems with the arm virt machine type. Has nobody tried this
use case with that? Is the use case something valid for riscv, but
not arm?
Firstly we have to enable pcie passthru on host, find the device groups,
e.g. the vga card, and add their pci ids to host kernel cmdline:
vfio-pci.ids=10de:0f02,10de:0e08
then start vm through qemu as follows:
$Q -machine virt -m 4G -smp 4 -nographic \
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \
-kernel ./vmlinuz -initrd initrd.img -append "root=/dev/vda1 rw" \
-drive
file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw
\
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-netdev user,id=vnet,hostfwd=:127.0.0.1:2223-:22 -device
virtio-net-pci,netdev=vnet
Without this patch, qemu exits immediately instead of boots up.
Just tried pcie passthru on arm, it cannot handle 4G memory either.
$Q -m 4G -smp 4 -cpu max -M virt -nographic \
-pflash /usr/share/AAVMF/AAVMF_CODE.fd -pflash flash1.img \
-drive if=none,file=ubuntu-22.04-server-cloudimg-arm64.img,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1
qemu-system-aarch64: -device vfio-pci,host=01:00.0: VFIO_MAP_DMA failed:
Invalid argument
qemu-system-aarch64: -device vfio-pci,host=01:00.0: vfio 0000:01:00.0:
failed to setup container for group 11: memory listener initialization
failed: Region mach-virt.ram: vfio_dma_map(0x55de3c2a97f0, 0x40000000,
0x100000000, 0x7f8fcbe00000) = -22 (Invalid argument)
Thanks,
Fei.
Thanks,
drew