[Qemu-devel] AArch64 QEMU System emulation: issue with TTBR0

[Claudio Fontana]

Hello Peter,

I am porting OSv to AArch64, and I have some working code running on
the Foundation Models,
where I run qemu natively with --enable-kvm,

which does not seem to work when run instead on top of the system emulation.

In particular I get a sync exception when I try to msr to TTBR0_EL1.

The ESR as read in env->cp15.esr_el[1] is 0x8400000e, which looking up
in the ESR table means

Instruction fault, with IFSC (instruction fault status code) = 0xe,
which should match

0b0011LL = permission fault (LL indicates level at which fault occurred).

with LL = 0b10 meaning EL2.

The code is in particular:

00000000401db2d0 <mmu::switch_to_runtime_page_tables()>:
    401db2d0:   d00037a0        adrp    x0, 408d1000 <unique_mtx+0x10>
    401db2d4:   9130e000        add     x0, x0, #0xc38
    401db2d8:   f9400000        ldr     x0, [x0]
    401db2dc:   92748c00        and     x0, x0, #0xfffffffff000
    401db2e0:   d5182000        msr     ttbr0_el1, x0
    401db2e4:   d5033fdf        isb
    401db2e8:   d00037a0        adrp    x0, 408d1000 <unique_mtx+0x10>
    401db2ec:   9130e000        add     x0, x0, #0xc38
    401db2f0:   f9400400        ldr     x0, [x0,#8]
    401db2f4:   92748c00        and     x0, x0, #0xfffffffff000
    401db2f8:   d5182020        msr     ttbr1_el1, x0
    401db2fc:   d5033fdf        isb
    401db300:   d5033f9f        dsb     sy
    401db304:   d508831f        tlbi    vmalle1is
    401db308:   d5033f9f        dsb     sy
    401db30c:   d5033fdf        isb
    401db310:   d65f03c0        ret

ELR_EL1 in env->elr_el[1] reads as 0x401da200, which is strangely
enough the address of the first instruction of the exception vector
entry for sync:

        ...
    401da200:   14000169        b       401da7a4 <entry_sync>
    401da204:   d503201f        nop
        ...
    401da280:   14000174        b       401da850 <entry_irq>
    401da284:   d503201f        nop

The source is available at:

https://github.com/cloudius-systems/osv/blob/master/arch/aarch64/mmu.cc

Thanks for any advice,

Claudio