[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy
From: |
Leonardo Bras Soares Passos |
Subject: |
Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy |
Date: |
Thu, 2 Sep 2021 04:22:55 -0300 |
Hello Daniel, thanks for the feedback !
On Tue, Aug 31, 2021 at 10:17 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Aug 31, 2021 at 08:02:39AM -0300, Leonardo Bras wrote:
> > Call qio_channel_set_zerocopy(true) in the start of every multifd thread.
> >
> > Change the send_write() interface of multifd, allowing it to pass down
> > flags for qio_channel_write*().
> >
> > Pass down MSG_ZEROCOPY flag for sending memory pages, while keeping the
> > other data being sent at the default copying approach.
> >
> > Signed-off-by: Leonardo Bras <leobras@redhat.com>
> > ---
> > migration/multifd-zlib.c | 7 ++++---
> > migration/multifd-zstd.c | 7 ++++---
> > migration/multifd.c | 9 ++++++---
> > migration/multifd.h | 3 ++-
> > 4 files changed, 16 insertions(+), 10 deletions(-)
>
> > @@ -675,7 +676,8 @@ static void *multifd_send_thread(void *opaque)
> > }
> >
> > if (used) {
> > - ret = multifd_send_state->ops->send_write(p, used,
> > &local_err);
> > + ret = multifd_send_state->ops->send_write(p, used,
> > MSG_ZEROCOPY,
> > + &local_err);
>
> I don't think it is valid to unconditionally enable this feature due to the
> resource usage implications
>
> https://www.kernel.org/doc/html/v5.4/networking/msg_zerocopy.html
>
> "A zerocopy failure will return -1 with errno ENOBUFS. This happens
> if the socket option was not set, the socket exceeds its optmem
> limit or the user exceeds its ulimit on locked pages."
You are correct, I unfortunately missed this part in the docs :(
> The limit on locked pages is something that looks very likely to be
> exceeded unless you happen to be running a QEMU config that already
> implies locked memory (eg PCI assignment)
Do you mean the limit an user has on locking memory?
If so, that makes sense. I remember I needed to set the upper limit of locked
memory for the user before using it, or adding a capability to qemu before.
Maybe an option would be trying to mlock all guest memory before setting
zerocopy=on in qemu code. If it fails, we can print an error message and fall
back to not using zerocopy (following the idea of a new io_async_writev()
I told you in the previous mail).
>
>
> Regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
>
Best regards,
Leonardo
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy,
Leonardo Bras Soares Passos <=
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Daniel P . Berrangé, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Leonardo Bras Soares Passos, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Daniel P . Berrangé, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Leonardo Bras Soares Passos, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Daniel P . Berrangé, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Leonardo Bras Soares Passos, 2021/09/02
- Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Dr. David Alan Gilbert, 2021/09/07
Re: [PATCH v1 3/3] migration: multifd: Enable zerocopy, Peter Xu, 2021/09/07