Hello,
I'm currently looking into implementing a 'file:' uri for migration save
in qemu. Ideally the solution will be O_DIRECT compatible. I'm aware of
the branch https://gitlab.com/berrange/qemu/-/tree/mig-file. In the
process of brainstorming how a solution would like the a couple of
questions transpired that I think warrant wider discussion in the
community.
First, implementing a solution which is self-contained within qemu would
be easy enough( famous last words) but the gist is one has to only care
about the format within qemu. However, I'm being told that what libvirt
does is prepend its own custom header to the resulting saved file, then
slipstreams the migration stream from qemu. Now with the solution that I
envision I intend to keep all write-related logic inside qemu, this
means there's no way to incorporate the logic of libvirt. The reason I'd
like to keep the write process within qemu is to avoid an extra copy of
data between the two processes (qemu outging migration and libvirt),
with the current fd approach qemu is passed an fd, data is copied
between qemu/libvirt and finally the libvirt_iohelper writes the data.
So the question which remains to be answered is how would libvirt make
use of this new functionality in qemu? I was thinking something along
the lines of :
1. Qemu writes its migration stream to a file, ideally on a filesystem
which supports reflink - xfs/btrfs
2. Libvirt writes it's header to a separate file
2.1 Reflinks the qemu's stream right after its header
2.2 Writes its trailer
3. Unlink() qemu's file, now only libvirt's file remains on-disk.
I wouldn't call this solution hacky though it definitely leaves some
bitter aftertaste.
Another solution would be to extend the 'fd:' protocol to allow multiple
descriptors (for multifd) support to be passed in. The reason dup()
can't be used is because in order for multifd to be supported it's
required to be able to write to multiple, non-overlapping regions of the
file. And duplicated fd's share their offsets etc. But that really seems
more or less hacky. Alternatively it's possible that pwrite() are used
to write to non-overlapping regions in the file. Any feedback is welcomed.
Regards,
Nikolay