Re: vnc clipboard support

[Daniel P . Berrangé]

On Thu, Jan 28, 2021 at 06:12:24PM +0100, Gerd Hoffmann wrote:
>   Hi folks,
> 
> I'm looking for a good way to implement cut+paste support for vnc.
> 
> The vnc core protocol has support for text/plain cut+paste, and there
> is an extension adding support for other formats.  That'll cover one
> part of the problem, exchanging cut+paste data between vnc client and
> qemu vnc server.

NB the VNC cut+paste impl is technically undefined  for non-7-bit ascii
data. In reality though I'd assume most servers/clients and up sending
UTF8. 

> The tricky part is the second: the guest <=> qemu communication.
> I see basically two possible approaches here:
> 
>   (1) Have some guest agent (spice does it that way).
>       Advantage: more flexible, allows more features.
>       Disadvantage: requires agent in the guest.



>   
>   (2) Send text as key events.
>       Advantage: no guest agent needed.
>       Disadvantage: is translated by guests keyboard map, so qemu
>       needs to know the map for proper char -> key event translation.
>       Only works for text/plain and only for chars you can easily
>       type, anything needing input methods (emoji 😊 for example)
>       isn't going to fly.
> 
> I think that (1) is clearly the better way.

I agree - the agent based approach also lets you have a bi-directional
solution. ie copy in the guest, paste in the host, which you can't
do by simulating key presses.  Simulating keypress is insanely slow
too, so useless for more than 100 bytes of text really.

>                                              Given that the agent
> would need to run in user wayland/xorg session context the existing
> qemu-guest-agent will not work.  Also linking against some UI library
> like gtk3 for clipboard handling is not something we want for the
> qemu-guest-agent.  So we need another one, I'll name it
> qemu-clipboard-agent for the rest of this mail.  And we need a
> communication channel.

I've never looked at the spice-guest-agent code, but I wonder if there
is any scope for re-using it with VNC, or is it too closely tangled
up with SPICE ?  The advantage of reuse is that all existing guest
OS with spice-guest-agent installed will "just work".

> I'd tend to model the qemu-clipboard-agent simliar to the
> qemu-guest-agent, i.e. have some stream as communication path and run
> some stream protocol over it.
> 
> Stream options I see are (in order of personal preference):
> 
>   (1) New virtio-serial port.  virtio-serial likely is there anyway
>       for the qemu-guest-agent ...
> 
>   (2) Have qemu-clipboard-agent and qemu-guest-agent share the agent
>       channel, i.e. qemu-clipboard-agent will proxy everything through
>       qemu-guest-agent (spice does it that way).
> 
> Protocol options I see are (not sure yet which to prefer, need to have
> a closer look at the candidates):
> 
>   (1) Add clipboard commands to QMP and use these.
> 
>   (2) Reuse the clipboard bits of the vnc protocol (forward
>       VNC_MSG_CLIENT_CUT_TEXT messages to the guest agent)
> 
>   (3) Reuse the clipboard bits of the spice-agent protocol.
> 
>   (4) Reuse the clipboard bits of the wayland protocol.
> 
> Once we have sorted the qemu <-> guest communication path it should be
> possible to also hook up other UIs (specifically gtk) without too much
> effort.  Which probably makes (2) a rather poor choice.

I think the VNC clipboard stuff is too crude to base a long term
design on. I figure we'll want to be at least inspired by (3) or
(4).

> Comments?
> Suggestions?
> Other ideas?

The QEMU guest agent is a single system level agent. IIUC, with SPICE
you have a single system level agent, and then zero or more session
level agents - one per desktop login basically. I assume we'll want
to support the latter too, at least from a design level, even if we
don't implemnet multi session straight away.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|