bug#31149: 27.0.50; (gui-get-selection nil 'text/html) returns mis-decoded text

[Eli Zaretskii]

Ping!

> Date: Sat, 14 Apr 2018 09:32:41 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: larsi@gnus.org, 31149@debbugs.gnu.org
> 
> > From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> > Date: Fri, 13 Apr 2018 16:55:26 -0400
> > Cc: Lars Ingebrigtsen <larsi@gnus.org>
> > 
> > (gui-get-selection nil 'text/html)
> > 
> > returns utf-16 text when the primary selection is owned by Mozilla, but
> > we decode it as latin-1 instead, so it looks like garbage.
> > 
> > I don't know why we're getting utf-16.  Is that what standards say it
> > should do?  If so, we should adjust our code (which currently knows
> > nothing about the `text/html` target-type).
> > 
> > As for why we decode it as latin-1, it's (under GNU/Linux; Lars may be
> > using something else because he's getting something with a `charset`
> > property which I don't get here) because:
> > - selection_data_to_lisp_data (in xselect.c) makes a unibyte string with
> >   the property `foreign-selection` set to `STRING` when the actual
> >   string type is not known (as opposed to COMPOUND-TEXT and
> >   UTF8-STRING, basically).
> > - in gui-get-selection we then have a mapping from `STRING` to
> >   `iso-8859-1` (which is apparently the right thing for the official
> >   `STRING` target-type in X11).
> > 
> > I can't figure out if/where these kinds of things about the X11
> > selection protocol is described, but at least in `xclip` they have
> > a hack specifically for this case:
> > 
> >     [...]
> >     if (html != None && sel_type == html) {
> >     /* if the buffer contains UCS-2 (UTF-16), convert to
> >      * UTF-8.  Mozilla-based browsers do this for the
> >      * text/html target.
> >      */
> >     [...]
> > 
> > and according to the subsequent code it's not even always the
> > same endianness.
> > 
> > I don't know what is the difference between the `target-type` passed to
> > x-get-selection-internal and the `foreign-selection` property we get on
> > the returned string (they seem to be the same in my tests, except when
> > the type is not one of the known ones, and where we then force
> > `foreign-selection` to be `STRING`).
> 
> I hope Handa-san (CC'ed) could comment on this.