Re: [PATCH v2 28/29] migration: Add direct-io parameter

[Daniel P . Berrangé]

On Tue, Oct 31, 2023 at 11:33:24AM -0300, Fabiano Rosas wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Tue, Oct 31, 2023 at 10:05:56AM -0300, Fabiano Rosas wrote:
> >> Daniel P. Berrangé <berrange@redhat.com> writes:
> >> 
> >> > On Mon, Oct 30, 2023 at 07:51:34PM -0300, Fabiano Rosas wrote:
> >> >> I could use some advice on how to solve this situation. The fdset code
> >> >> at monitor/fds.c and the add-fd command don't seem to be usable outside
> >> >> the original use-case of passing fds with different open flags.
> >> >> 
> >> >> There are several problems, the biggest one being that there's no way to
> >> >> manipulate the set of file descriptors aside from asking for duplication
> >> >> of an fd that matches a particular set of flags.
> >> >> 
> >> >> That doesn't work for us because the two fds we need (one for main
> >> >> channel, other for secondary channels) will have the same open flags. So
> >> >> the fdset code will always return the first one it finds in the set.
> >> >
> >> > QEMU may want multiple FDs *internally*, but IMHO that fact should
> >> > not be exposed to mgmt applications. It would be valid for a QEMU
> >> > impl to share the same FD across multiple threads, or have a different
> >> > FD for each thread. All threads are using pread/pwrite, so it is safe
> >> > for them to use the same FD if they desire. It is a private impl choice
> >> > for QEMU at any given point in time and could change over time.
> >> >
> >> 
> >> Sure, I don't disagree. However up until last week we had a seemingly
> >> usable "add-fd" command that allows the user to provide a *set of file
> >> descriptors* to QEMU. It's just now that we're learning that interface
> >> serves only a special use-case.
> >
> > AFAICT though we don't need add-fd to support passing many files
> > for our needs. Saving only requires a single FD. All others can
> > be opened by dup(), so the limitation of add-fd is irrelevant
> > surely ?
> 
> Only once we decide to use one FD. If we had a generic add-fd backend,
> then that's already a user-facing API, so the "implementation detail"
> argument becomes weaker.
> 
> With a single FD we'll need to be very careful about what code is
> allowed to run while the multifd channels are doing IO. Since O_DIRECT
> is not widely supported, now we have to also be careful about someone
> using that QEMUFile handle to do unaligned writes and not even noticing
> that it breaks direct IO. None of this in unworkable, of course, I just
> find the design way clearer with just the file name + offset.

I guess I'm not seeing the problem still.  A single FD is passed across
from libvirt, but QEMU is free to turn that into *many* FDs for its
internal use, using dup() and then setting O_DIRECT on as many/few of
the dup()d FDs as its wants to.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|