KVM guests reading/writing guest memory directly and accurately

[Berto Furth]

Hi qemu-arm,

Can anyone give me some advice on how a machine or device can read and write 
kvm guest ram memory and get a guaranteed up to date result? Can someone point 
me at an example in the latest QEMU source code? I'm working with an ARM-32 
guest (-cpu host,aarch64=off) running on an ARM-64 host (Cortex A72 - Raspberry 
Pi4b). 

I have a problem where if I write directly to my guest RAM, (such as a DMA 
transfer) then the guest can't see the change straight away. Similarly when the 
host writes memory, the guest doesn't see the change until much later.

If during a KVM_EXIT_MMIO the qemu host changes some values in guest ram memory 
(via address_space_write() or cpu_physical_memory_rw() etc...) , is there a way 
to make the guest be able to accurately read that memory as soon as the exit is 
complete. Additionally if a guest changes a value in ram just before a 
KVM_EXIT_MMIO, is there a way to ensure that the QEMU host can then read the up 
to date newly set values?

Is there a function I can call to get any memory caches sitting in between the 
QEMU host program and the guest to be flushed? 

Should a region of guest ram be set up any differently than calling the normal 
functions like memory_region_init_ram() and memory_region_add_subregion() to 
facilitate this? I see that there are some memory initialization functions that 
deal with logging dirty memory but from what I can see they're only used by VGA 
displays and for VM migration.

I understand that the proper thing to do is to set up the memory region in 
question as MMIO so that when the guest accesses this memory a KVM_EXIT_MMIO 
will occur but the memory region in question has to be executable and MMIO 
memory areas are not executable in QEMU. In addition it's not easily possible 
to predict before hand exactly what memory addresses are going to be involved 
in DMA, so I'd prefer to avoid having to dynamically construct little MMIO 
memory islands inside the main guest ram space as the guest runs.

I'm assuming that the guest could be modified to disable d-caching (modify the 
ARM register SCTLR / p15 ?) and that may help but I'm desperately trying to 
avoid that if possible because I'm working with a proprietary "blob" on the 
guest that I don't have all the source code for. 

I know it's not very professional of me to make an emotional plea, but I've 
been working on this for weeks and I am desperately hoping someone can point to 
a solution for me. I am not a KVM expert and so I'm hoping I'm just missing 
something simple and obvious that one of you can quickly point out for me. If I 
have missed something obvious then I'm more than open to taking your abuse, 
mockery, insults and derision in exchange for some guidance!!

Thanks so much for your help!