Hello Michael,
On 2/1/23 12:20, Michael S. Tsirkin wrote:
On Wed, Feb 01, 2023 at 12:14:18PM +0100, Maxime Coquelin wrote:
Thanks Eugenio for working on this.
On 1/31/23 20:10, Eugenio Perez Martin wrote:
Hi,
The current approach of offering an emulated CVQ to the guest and map
the commands to vhost-user is not scaling well:
* Some devices already offer it, so the transformation is redundant.
* There is no support for commands with variable length (RSS?)
We can solve both of them by offering it through vhost-user the same
way as vhost-vdpa do. With this approach qemu needs to track the
commands, for similar reasons as vhost-vdpa: qemu needs to track the
device status for live migration. vhost-user should use the same SVQ
code for this, so we avoid duplications.
One of the challenges here is to know what virtqueue to shadow /
isolate. The vhost-user device may not have the same queues as the
device frontend:
* The first depends on the actual vhost-user device, and qemu fetches
it with VHOST_USER_GET_QUEUE_NUM at the moment.
* The qemu device frontend's is set by netdev queues= cmdline parameter in qemu
For the device, the CVQ is the last one it offers, but for the guest
it is the last one offered in config space.
To create a new vhost-user command to decrease that maximum number of
queues may be an option. But we can do it without adding more
commands, remapping the CVQ index at virtqueue setup. I think it
should be doable using (struct vhost_dev).vq_index and maybe a few
adjustments here and there.
Thoughts?
I am fine with both proposals.
I think index remapping will require a bit more rework in the DPDK
Vhost-user library, but nothing insurmountable.
I am currently working on a PoC adding support for VDUSE in the DPDK
Vhost library, and recently added control queue support. We can reuse it
if we want to prototype your proposal.
Maxime
Thanks!
technically backend knows how many vqs are there, last one is cvq...
not sure we need full blown remapping ...
Before VHOST_USER_PROTOCOL_F_STATUS was supported by qemu (very
recently, v7.2.0), we had no way for the backend to be sure the
frontend won't configure new queue pairs, this not not defined in the
spec AFAICT [0]. In DPDK Vhost library, we notify the application it can
start to use the device once the first queue pair is setup and enabled,
then we notify the application when new queues are ready to be
processed. In this case, I think we cannot deduce whether the queue is a
data or a control queue when it is setup.
When VHOST_USER_PROTOCOL_F_STATUS is supported, we know no more queues
will be configured once the DRIVER_OK status is set. In this case, we
can deduce the last queue setup will be the control queue at DRIVER_OK
time if the number of queues is odd.
Using index remapping, we would know directly at queue setup time
whether this is a data or control queue based on its index value,
i.e. if the index equals to max queue index supported by the backend.
But thinking at it again, we may at least back this with a protocol
feature to avoid issues with legacy backends.
I hope it clarifies, let me know if anything unclear.
Thanks,
Maxime
[0]:
https://elixir.bootlin.com/qemu/latest/source/docs/interop/vhost-user.rst