Re: [Qemu-devel] AArch64 QEMU System emulation: issue with TTBR0

[Claudio Fontana]

On 2 June 2014 18:16, Claudio Fontana <address@hidden> wrote:
> Hello Peter,
>
> I am porting OSv to AArch64, and I have some working code running on
> the Foundation Models,
> where I run qemu natively with --enable-kvm,
>
> which does not seem to work when run instead on top of the system emulation.
>
> In particular I get a sync exception when I try to msr to TTBR0_EL1.
>
> The ESR as read in env->cp15.esr_el[1] is 0x8400000e, which looking up
> in the ESR table means
>
> Instruction fault, with IFSC (instruction fault status code) = 0xe,
> which should match
>
> 0b0011LL = permission fault (LL indicates level at which fault occurred).
>
> with LL = 0b10 meaning EL2.
>
> The code is in particular:
>
> 00000000401db2d0 <mmu::switch_to_runtime_page_tables()>:
>     401db2d0:   d00037a0        adrp    x0, 408d1000 <unique_mtx+0x10>
>     401db2d4:   9130e000        add     x0, x0, #0xc38
>     401db2d8:   f9400000        ldr     x0, [x0]
>     401db2dc:   92748c00        and     x0, x0, #0xfffffffff000
>     401db2e0:   d5182000        msr     ttbr0_el1, x0
>     401db2e4:   d5033fdf        isb
>     401db2e8:   d00037a0        adrp    x0, 408d1000 <unique_mtx+0x10>
>     401db2ec:   9130e000        add     x0, x0, #0xc38
>     401db2f0:   f9400400        ldr     x0, [x0,#8]
>     401db2f4:   92748c00        and     x0, x0, #0xfffffffff000
>     401db2f8:   d5182020        msr     ttbr1_el1, x0
>     401db2fc:   d5033fdf        isb
>     401db300:   d5033f9f        dsb     sy
>     401db304:   d508831f        tlbi    vmalle1is
>     401db308:   d5033f9f        dsb     sy
>     401db30c:   d5033fdf        isb
>     401db310:   d65f03c0        ret
>
> ELR_EL1 in env->elr_el[1] reads as 0x401da200, which is strangely
> enough the address of the first instruction of the exception vector
> entry for sync:
>
>         ...
>     401da200:   14000169        b       401da7a4 <entry_sync>
>     401da204:   d503201f        nop
>         ...
>     401da280:   14000174        b       401da850 <entry_irq>
>     401da284:   d503201f        nop
>
> The source is available at:
>
> https://github.com/cloudius-systems/osv/blob/master/arch/aarch64/mmu.cc
>
> Thanks for any advice,
>
> Claudio

hmm one thing that came to mind of course after posting, is that QEMU
in system emulation mode probably tries to start at EL2 or EL3, while
on OSv I assume to be running as a guest at EL1.
However, I have a check to validate the exception level, after which I
should drop to EL1 instead of halting if I am at EL2/EL3.

The check however does not seem to capture this case, as it passes.
The check goes:

validate_el:
        mrs     x0, currentel
        ubfm    x0, x0, #2, #3 // current EL[3:2] -> X0
        cmp     x0, #1
        b.ne    halt
        ret