[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH 00/12] [RFC PATCHv2] Add vhost-user Video decode
From: |
Albert Esteve |
Subject: |
[PATCH 00/12] [RFC PATCHv2] Add vhost-user Video decode |
Date: |
Wed, 22 Mar 2023 15:21:20 +0100 |
v1: https://patchew.org/QEMU/20211209145601.331477-1-peter.griffin@linaro.org/
v1 -> v2:
- Address the feedback found in the original patch
- Rebase patch so that it applies to latest master
- Add PoC implementation for dmabuf sharing (using udmabuf for creation)
- Test the patch on a real HW codec
This series adds support for virtio-video decoder devices in Qemu
and also provides a vhost-user-video vmm implementation.
The vhost-user-video vmm currently parses virtio-video v3 protocol
(as that still was what the Linux frontend driver prototype implemented
at the timei, we kept working over this).
It then converts that to a v4l2 mem2mem stateful decoder device.
Currently, this has been tested using v4l2 vicodec test driver in Linux [1],
and also with a real HW codec, i.e., QCom Venus driver, which implements
a v4l2 stateful decoder/encoder. In order to support vicodec driver, the
virtio backend supports the FWHT format.
The primary goal so far has been to allow testing with Qemu. Using
vicodec on the host allows a purely virtual dev env, and for
CI integration in the future by kernelci, etc.
This series adds the virtio_video.h header for testing purposes,
but the driver code has not been upstreamed yet, as the specs
are still under discussion.
Therefore, the commit does not need to be reviewed at this stage.
This version includes a proof of concept implementation of dma-buf
sharing mechanism. Since the infrastructure is not yet available in Qemu,
we employed udmabuf device to create new dma-buf structures when the
driver asks for a shared object memory stream, and then we copy the
contents of the shared memory buffer into the newly created buffer.
Also, note that this PoC was necessary for making the patch work with
QC Venus driver as it does not support USERPTR memory in its latest
release. In any case, a thorough review of these two commits is not
necessary in its current stage, as the implementation is subjected
to big changes.
We tested this VMM using v4l2-ctl from v4l2 utils in the guest
to do a video decode to a file. This can then be validated using ffplay
v4l2-compliance tool in the guest has also been run which stresses the
interface and issues lots of syscall level tests.
See the vhost-user-video.rst for example commands on how to configure
guest kernel and do a video decode using Qemu, vicodec using this VMM.
Next steps:
Recently there was a big update for virtio-video specs being discussed
in the virtio-dev mailing list [2], with a proposal for moving the specs
closer to the v4l2 framework.
Following these news, the next step for this series is to update the
implementation to the new specs approach and join efforts in verifying
the driver implementation [3], and the specs themselves.
Linux virtio-video frontend driver code:
https://github.com/aesteve-rh/linux
(adds FWHT support on top of https://github.com/Gnurou/linux)
Qemu vmm code:
https://github.com/aesteve-rh/qemu/tree/virtio_video_v3
Applies cleanly to:
git://git.qemu.org/qemu.git master(c283ff89d11ff123efc9af49128ef58511f73012)
[1] https://lwn.net/Articles/760650/
[2] https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg09126.html
[3]
https://github.com/Gnurou/linux/blob/virtio-v4l2/drivers/media/virtio-v4l2/virtio_v4l2_driver.c
Albert Esteve (5):
vhost-user.json: add video type
tests/qtest: add virtio-video test
vhost-user-video: add dev_type to CLI
vhost-user-video-udmabuf: add udmabuf helpers
Add support for v4l2_memory_dmabuf
Peter Griffin (7):
docs: Add a vhost-virtio-video rst file
MAINTAINERS: Add virtio-video section
vhost-user-video: boiler plate code for vhost-user-video device
vhost-user-video: add meson subdir build logic
standard-headers: Add virtio_video.h
hw/display: add vhost-user-video-pci
tools/vhost-user-video: Add initial vhost-user-video vmm
MAINTAINERS | 8 +
docs/interop/vhost-user.json | 2 +
docs/system/devices/vhost-user-video.rst | 124 ++
hw/display/Kconfig | 5 +
hw/display/meson.build | 3 +
hw/display/vhost-user-video-pci.c | 82 +
hw/display/vhost-user-video.c | 419 ++++
include/hw/virtio/vhost-user-video.h | 43 +
include/standard-headers/linux/virtio_video.h | 513 +++++
tests/qtest/libqos/meson.build | 1 +
tests/qtest/libqos/virtio-video.c | 179 ++
tests/qtest/libqos/virtio-video.h | 39 +
tests/qtest/vhost-user-test.c | 32 +
tools/meson.build | 7 +
tools/vhost-user-video/50-qemu-video.json.in | 5 +
tools/vhost-user-video/meson.build | 10 +
tools/vhost-user-video/v4l2_backend.c | 1838 +++++++++++++++++
tools/vhost-user-video/v4l2_backend.h | 111 +
tools/vhost-user-video/vhost-user-video.c | 1787 ++++++++++++++++
tools/vhost-user-video/virtio_video_helpers.c | 466 +++++
tools/vhost-user-video/virtio_video_helpers.h | 198 ++
tools/vhost-user-video/virtio_video_udmabuf.c | 180 ++
tools/vhost-user-video/vuvideo.h | 51 +
23 files changed, 6103 insertions(+)
create mode 100644 docs/system/devices/vhost-user-video.rst
create mode 100644 hw/display/vhost-user-video-pci.c
create mode 100644 hw/display/vhost-user-video.c
create mode 100644 include/hw/virtio/vhost-user-video.h
create mode 100644 include/standard-headers/linux/virtio_video.h
create mode 100644 tests/qtest/libqos/virtio-video.c
create mode 100644 tests/qtest/libqos/virtio-video.h
create mode 100644 tools/vhost-user-video/50-qemu-video.json.in
create mode 100644 tools/vhost-user-video/meson.build
create mode 100644 tools/vhost-user-video/v4l2_backend.c
create mode 100644 tools/vhost-user-video/v4l2_backend.h
create mode 100644 tools/vhost-user-video/vhost-user-video.c
create mode 100644 tools/vhost-user-video/virtio_video_helpers.c
create mode 100644 tools/vhost-user-video/virtio_video_helpers.h
create mode 100644 tools/vhost-user-video/virtio_video_udmabuf.c
create mode 100644 tools/vhost-user-video/vuvideo.h
--
2.39.2
- [PATCH 00/12] [RFC PATCHv2] Add vhost-user Video decode,
Albert Esteve <=
- [PATCH 01/12] docs: Add a vhost-virtio-video rst file, Albert Esteve, 2023/03/22
- [PATCH 02/12] MAINTAINERS: Add virtio-video section, Albert Esteve, 2023/03/22
- [PATCH 04/12] vhost-user-video: add meson subdir build logic, Albert Esteve, 2023/03/22
- [PATCH 05/12] standard-headers: Add virtio_video.h, Albert Esteve, 2023/03/22
- [PATCH 06/12] hw/display: add vhost-user-video-pci, Albert Esteve, 2023/03/22
- [PATCH 03/12] vhost-user-video: boiler plate code for vhost-user-video device, Albert Esteve, 2023/03/22
- [PATCH 07/12] vhost-user.json: add video type, Albert Esteve, 2023/03/22
- [PATCH 09/12] tests/qtest: add virtio-video test, Albert Esteve, 2023/03/22