[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v11 0/7] Support x2APIC mode with TCG accelerator
From: |
Michael S. Tsirkin |
Subject: |
Re: [PATCH v11 0/7] Support x2APIC mode with TCG accelerator |
Date: |
Tue, 26 Dec 2023 04:21:17 -0500 |
On Mon, Dec 25, 2023 at 11:40:54PM +0700, Bui Quang Minh wrote:
> Hi everyone,
>
> This series implements x2APIC mode in userspace local APIC and the
> RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu
> and AMD iommu are adjusted to support x2APIC interrupt remapping. With this
> series, we can now boot Linux kernel into x2APIC mode with TCG accelerator
> using either Intel or AMD iommu.
>
> Testing to boot my own built Linux 6.3.0-rc2, the kernel successfully boot
> with enabled x2APIC and can enumerate CPU with APIC ID 257
>
> Using Intel IOMMU
>
> qemu/build/qemu-system-x86_64 \
> -smp 2,maxcpus=260 \
> -cpu qemu64,x2apic=on \
> -machine q35 \
> -device intel-iommu,intremap=on,eim=on \
> -device qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
> -m 2G \
> -kernel $KERNEL_DIR \
> -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
> net.ifnames=0" \
> -drive file=$IMAGE_DIR,format=raw \
> -nographic \
> -s
>
> Using AMD IOMMU
>
> qemu/build/qemu-system-x86_64 \
> -smp 2,maxcpus=260 \
> -cpu qemu64,x2apic=on \
> -machine q35 \
> -device amd-iommu,intremap=on,xtsup=on \
> -device qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
> -m 2G \
> -kernel $KERNEL_DIR \
> -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
> net.ifnames=0" \
> -drive file=$IMAGE_DIR,format=raw \
> -nographic \
> -s
>
> Testing the emulated userspace APIC with kvm-unit-tests, disable test
> device with this patch
Seems to break build for windows/amd64
https://gitlab.com/mstredhat/qemu/-/pipelines/1118886361/failures
> diff --git a/lib/x86/fwcfg.c b/lib/x86/fwcfg.c
> index 1734afb..f56fe1c 100644
> --- a/lib/x86/fwcfg.c
> +++ b/lib/x86/fwcfg.c
> @@ -27,6 +27,7 @@ static void read_cfg_override(void)
>
> if ((str = getenv("TEST_DEVICE")))
> no_test_device = !atol(str);
> + no_test_device = true;
>
> if ((str = getenv("MEMLIMIT")))
> fw_override[FW_CFG_MAX_RAM] = atol(str) * 1024 * 1024;
>
> ~ env QEMU=/home/minh/Desktop/oss/qemu/build/qemu-system-x86_64 ACCEL=tcg \
> ./run_tests.sh -v -g apic
>
> TESTNAME=apic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/apic.flat -smp 2
> -cpu qemu64,+x2apic,+tsc-deadline -machine kernel_irqchip=split FAIL
> apic-split (54 tests, 8 unexpected failures, 1 skipped)
> TESTNAME=ioapic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/ioapic.flat -smp
> 1 -cpu qemu64 -machine kernel_irqchip=split PASS ioapic-split (19 tests)
> TESTNAME=x2apic TIMEOUT=30 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu
> qemu64,+x2apic,+tsc-deadline FAIL x2apic (54 tests, 8 unexpected failures,
> 1 skipped) TESTNAME=xapic TIMEOUT=60 ACCEL=tcg ./x86/run x86/apic.flat -smp
> 2 -cpu qemu64,-x2apic,+tsc-deadline -machine pit=off FAIL xapic (43 tests,
> 6 unexpected failures, 2 skipped)
>
> FAIL: apic_disable: *0xfee00030: 50014
> FAIL: apic_disable: *0xfee00080: f0
> FAIL: apic_disable: *0xfee00030: 50014
> FAIL: apic_disable: *0xfee00080: f0
> FAIL: apicbase: relocate apic
>
> These errors are because we don't disable MMIO region when switching to
> x2APIC and don't support relocate MMIO region yet. This is a problem
> because, MMIO region is the same for all CPUs, in order to support these we
> need to figure out how to allocate and manage different MMIO regions for
> each CPUs. This can be an improvement in the future.
>
> FAIL: nmi-after-sti
> FAIL: multiple nmi
>
> These errors are in the way we handle CPU_INTERRUPT_NMI in core TCG.
>
> FAIL: TMCCT should stay at zero
>
> This error is related to APIC timer which should be addressed in separate
> patch.
>
> Version 11 changes,
> - Patch 2:
> + Rebase to master and fix conflict with commit c04cfb4596 (hw/i386: fix
> short-circuit logic with non-optimizing builds)
>
> Version 10 changes,
> - Patch 2:
> + Fix null pointer dereference due to uninitialized local_apics when using
> machine none
> - Patch 5, 7:
> + These patches are added to follow the bios-tables-test instructions to
> commit the new changed IVRS.ivrs binary file
>
> Version 9 changes,
> - Patch 1:
> + Create apic_msr_read/write which is a small wrapper around
> apic_register_read/write that have additional x2apic mode check
> - Patch 2:
> + Remove raise_exception_ra which is is TCG specific. Instead, return -1
> and let the accelerator raise the appropriate exception
> + Refactor apic_get_delivery_bitmask a little bit to reduce line length
> + Move cpu_has_x2apic_feature and cpu_set_apic_feature from patch 3 to
> patch 2 so that patch 2 can be compiled without patch 3
> - Patch 3:
> + set_base in APICCommonClass now returns an int to indicate error
> + Remove raise_exception_ra in apic_set base which is is TCG specific.
> Instead, return -1 and let the accelerator raise the appropriate
> exception
>
> Version 8 changes,
> - Patch 2, 4:
> + Rebase to master and resolve conflicts in these 2 patches
>
> Version 7 changes,
> - Patch 4:
> + If eim=on, keep checking if kvm x2APIC is enabled when kernel-irqchip
> is split
>
> Version 6 changes,
> - Patch 5:
> + Make all places use the amdvi_extended_feature_register to get extended
> feature register
>
> Version 5 changes,
> - Patch 3:
> + Rebase to master and fix conflict
> - Patch 5:
> + Create a helper function to get amdvi extended feature register instead
> of storing it in AMDVIState
>
> Version 4 changes,
> - Patch 5:
> + Instead of replacing IVHD type 0x10 with type 0x11, export both types
> for backward compatibility with old guest operating system
> + Flip the xtsup feature check condition in amdvi_int_remap_ga for
> readability
>
> Version 3 changes,
> - Patch 2:
> + Allow APIC ID > 255 only when x2APIC feature is supported on CPU
> + Make physical destination mode IPI which has destination id 0xffffffff
> a broadcast to xAPIC CPUs
> + Make cluster address 0xf in cluster model of xAPIC logical destination
> mode a broadcast to all clusters
> + Create new extended_log_dest to store APIC_LDR information in x2APIC
> instead of extending log_dest for backward compatibility in vmstate
>
> Version 2 changes,
> - Add support for APIC ID larger than 255
> - Adjust AMD iommu for x2APIC support
> - Reorganize and split patch 1,2 into patch 1,2,3 in version 2
>
> Thanks,
> Quang Minh.
>
> Bui Quang Minh (7):
> i386/tcg: implement x2APIC registers MSR access
> apic: add support for x2APIC mode
> apic, i386/tcg: add x2apic transitions
> intel_iommu: allow Extended Interrupt Mode when using userspace APIC
> test: bios-tables-test: prepare IVRS change in ACPI table
> amd_iommu: report x2APIC support to the operating system
> test: bios-tables-test: add IVRS changed binary
>
> hw/i386/acpi-build.c | 129 +++++---
> hw/i386/amd_iommu.c | 29 +-
> hw/i386/amd_iommu.h | 16 +-
> hw/i386/intel_iommu.c | 6 +-
> hw/i386/kvm/apic.c | 3 +-
> hw/i386/x86.c | 6 +-
> hw/i386/xen/xen_apic.c | 3 +-
> hw/intc/apic.c | 473 +++++++++++++++++++++------
> hw/intc/apic_common.c | 22 +-
> hw/intc/trace-events | 4 +-
> include/hw/i386/apic.h | 8 +-
> include/hw/i386/apic_internal.h | 9 +-
> target/i386/cpu-sysemu.c | 18 +-
> target/i386/cpu.c | 9 +-
> target/i386/cpu.h | 9 +
> target/i386/tcg/sysemu/misc_helper.c | 41 ++-
> target/i386/whpx/whpx-apic.c | 3 +-
> tests/data/acpi/q35/IVRS.ivrs | Bin 104 -> 176 bytes
> 18 files changed, 600 insertions(+), 188 deletions(-)
>
> --
> 2.25.1
- [PATCH v11 0/7] Support x2APIC mode with TCG accelerator, Bui Quang Minh, 2023/12/25
- [PATCH v11 1/7] i386/tcg: implement x2APIC registers MSR access, Bui Quang Minh, 2023/12/25
- [PATCH v11 2/7] apic: add support for x2APIC mode, Bui Quang Minh, 2023/12/25
- [PATCH v11 3/7] apic, i386/tcg: add x2apic transitions, Bui Quang Minh, 2023/12/25
- [PATCH v11 4/7] intel_iommu: allow Extended Interrupt Mode when using userspace APIC, Bui Quang Minh, 2023/12/25
- [PATCH v11 5/7] test: bios-tables-test: prepare IVRS change in ACPI table, Bui Quang Minh, 2023/12/25
- [PATCH v11 6/7] amd_iommu: report x2APIC support to the operating system, Bui Quang Minh, 2023/12/25
- [PATCH v11 7/7] test: bios-tables-test: add IVRS changed binary, Bui Quang Minh, 2023/12/25
- Re: [PATCH v11 0/7] Support x2APIC mode with TCG accelerator,
Michael S. Tsirkin <=