Re: A brief look at deprecating our JSON extensions over RFC 8259

[Daniel P . Berrangé]

On Mon, Feb 22, 2021 at 03:57:22PM +0100, Markus Armbruster wrote:
> We use JSON in several external interfaces:
> 
> * QMP
> 
> * The guest agent's QMP
> 
> * QAPIfied command line options when the option argument starts with
>   '{'
> 
> * The block layer's pseudo-protocol "json:" (which can get embedded in
>   image headers)
> 
> I *think* that's all.
> 
> The JSON parser we use for these interfaces supports extensions over RFC
> 8259.  Quoting json-lexer.c:
> 
>     - Extra escape sequence in strings:
>       0x27 (apostrophe) is recognized after escape, too
> 
>     - Single-quoted strings:
>       Like double-quoted strings, except they're delimited by %x27
>       (apostrophe) instead of %x22 (quotation mark), and can't contain
>       unescaped apostrophe, but can contain unescaped quotation mark.
> 
>     - Interpolation, if enabled:
>       The lexer accepts %[A-Za-z0-9]*, and leaves rejecting invalid
>       ones to the parser.
> 
> Ignore interpolation; it's never enabled at external interfaces.
> 
> This leaves single-quotes strings and the escape sequence to go with
> them.
> 
> I disabled them as an experiment.  Some 20 iotests, a qtest and two unit
> tests explode.
> 
> The unit test testing the JSON parser is of course excused.
> 
> The remaining qtest and the unit test could perhaps be dismissed as
> atypical use of QEMU from C.  The iotests less so, I think.
> 
> I looked at some iotest failures, and quickly found single-quoted
> strings used with all external interfaces except for qemu-ga's QMP.
> 
> We could certainly tidy up the tests to stick to standard JSON.
> However, the prevalence of single-quoted strings in iotests makes me
> suspect that they are being used in the field as well.  Deprecating the
> extension is likely more trouble than it's worth.

The shell based iotests use single quotes primarily because they're
being written in a language which lacks the concept of libraries and
and so all JSON is constructed by string substitution. Using single
quotes is convenient to avoid continually escaping double quotes.

For any other language constructing JSON documents through string
substitution is insanity, because they all have JSON libraries
available which let you construct JSON documents progamatically
without risk of introducing security flaws through malicious
substitutions.

This problem isn't unique to QEMU. Any app using JSON from the
shell will have the tedium of quote escaping. JSON is incredibly
widespread and no other apps felt it neccessary to introduce single
quoting support, because the benefit doesn't outweigh the interop
problem it introduces.

> Opinions?

IMHO we should deprecate and eventually remove single quotes. We
should expect mgmt apps to be using a JSON library to talk to QEMU
in general if they are using QMP. Sure some may be using shell, but
I'd expect that to be relatively few. Adapting is tedious but not
especially hard.

It would be nice at some point in future to have the option of using
a standard JSON library in part or all of QEMU. Especially if we
ever want to be able to have parts of QEMU written in non-C language,
we don't want to re-invent a custom JSON parser as the first step,
for back compatibility.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|