Re: [RFC PATCH v1 0/8] qapi: add generator for Golang interface

[Victor Toso]

Hi,

On Wed, May 11, 2022 at 04:17:35PM +0200, Markus Armbruster wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> > Caller
> >
> >     block_resize(device="dev0", size=1*GiB)
> >     block_resize(node_name="devnode0", size=1*GiB)
> >
> >
> > In golang definition
> >
> >    type BlockResizeArguments struct {
> >        Device string
> >        NodeName string
> >        Size int
> >    }
> >
> > Caller choice of
> >
> >    cmd := &BlockResizeCommand{
> >        Device: "dev0",
> >        Size: 1 * GiB,
> >    }
> >
> >    cmd := &BlockResizeCommand{
> >        NodeName: "devnode0",
> >        Size: 1 * GiB,
> >    }
> 
> Note that the Go bindings you sketched effectively use (poor
> man's) keyword arguments.
> 
> > Neither case can easily prevent passing Device and NodeName
> > at same time.
> 
> That defect lies at the schema's feet.

Right. The schema does not provide any metadata to explicit say
that only @device or @node-name should be used, correct?

This would be important to differentiate of a simple 'adding a
new optional argument' plus 'making this other argument
optional'.

> >> * At some future date, the old way gets deprecated: argument @device
> >>   acquires feature @deprecated.
> >
> > Ok, no change needed to the APIs in either case. Possibly have
> > code emit a warning if a deprecated field is set.
> >
> >> * Still later, the old way gets removed: @device is deleted, and
> >>   @node-name becomes mandatory.
> >
> > Again no change needed to APIs, but QEMU will throw back an
> > error if the wrong one is used. 
> >
> >> What is the proper version-spanning interface?
> >> 
> >> I figure it's both arguments optional, must specify the right one for
> >> the version of QEMU actually in use.  This spans versions, but it fails
> >> to abstract from them.
> >
> > Yep, I think that's inevitable in this scenario. THe plus side
> > is that apps that want to span versions can do so. The downside
> > is that apps that don't want smarts to span version, may loose
> > compile time warnings about use of the now deleted field. 
> 
> The version-spanning interface will arguably be a bad interface for any
> version.
>
> > I suggested the code generator have an option to say what level
> > of compat to use for generated code, so that apps can request an
> > API without compat, which will result in compile errors. This
> > though assumes every consumer app is embedding their own
> > generated copy of the code. Not neccessarily desirable.
> >
> > At the C level you can play games with __deprecated__ to get
> > compile time warnings in some cases. 
> >
> > #define GLIB_VERSION_MIN_REQUIRED GLIB_VERSION_2_56
> >
> > causes QEMU to get compile time warnings (or errors) if it
> > attempts to use a API feature deprecated in 2.56, even if
> > the API exists in the header & library. 
> >
> >
> >> Note that it's not enough to replace "delete member" by "mark member
> >> deleted in <version>".  You also have to keep full history for "is it
> >> optional".  And for types, because those can evolve compatibly, too,
> >> e.g. from struct to flat union, or from string to alternate of string
> >> and something else.  What is the proper version-spanning interface in
> >> all the possible cases?
> >
> > I've not thought through all possible scenarios, but there may end
> > up being restrictions, such that changes that were previously possible
> > may have to be forbidden.
> 
> "There may be restrictions" is not exactly a confidence-inspring design
> assumption.  We need a reasonably dependable idea on what exactly we're
> intending to sacrifice.

I can't help much here but I guess we can evolve QAPI schema as
we move forward. Adding metadata that helps document changes to
the benefit of giving code generators tools to provide a way to
work with those QAPI changes seems desirable, no?

> > One example,  in the past we could do deprecate a field 'foo', then
> > delete 'foo' and then some time re-introduce 'foo' with a completely
> > different type. That would not be possible if we wanted to maintain
> > compat, but in this example that's probably a good thing, as it'll
> > be super confusing to have the same field name change type like that
> > over time. Easier to just use a different name.
> >
> > So the question to me is not whether all our previous changes are
> > still possible, but whether enough of the typwes of change are
> > possible, such that we can cope with the ongoing maint in a
> > reasonable way. I don't think we've explored the possibility enough
> > to say one way or the other.
> >
> >> > Apps that wish to have version compat, would of course need to write
> >> > their code to be aware of which fields they need to seend for which
> >> > QEMU version.
> >> 
> >> At which point we're reinventing libvirt.

IMHO, at this moment, qapi-go is targeting communicating with
QEMU and handling multiple QEMU versions seems reasonable to me.

Perhaps libvirt can use qapi-go in the future or other generated
interface. That would be cool.

> > The premise of the code generators is that there are applications
> > that want to consume QEMU that are not libvirt. With this line of
> > reasoning we could easily say that all such applications should
> > just use libvirt and then we don't need to provide any of these
> > code generators.  The fact that we're considering these code
> > generators though, says that we're accepting there are valid use
> > cases that don't want to use libvirt for whatever reasons.
> 
> Can't resist any longer: "What has libvirt ever done for us?"
> https://www.youtube.com/watch?v=Qc7HmhrgTuQ
> 
> >                                                            It is
> > reasonable that some of those applications may wish to target
> > a wide range of QEMU versions, just like libvirt does.
> 
> At which point they're comitting to reinventing the relevant parts of
> libvirt.
> 
> I'd expect marshalling and umarshalling QMP to be among the
> smaller sub-problems then.  It may look relatively large at
> first, because it's among the first ones to be solved.  More so
> when you hand-wave away the more interesting ones of
> *abstraction* until they bite you in the posterior.

I might have missed it but I don't see unsolvable problems.

 1) We decide if we want a Golang interface that can communicate
    with multiple QEMU versions or not;
 2) We discuss how that Golang interface would look like;
 3) We discuss what is needed in QEMU/QAPI to achieve (2)
 4) We work on QEMU/QAPI to address (3)
 5) We move on with qapi-go proposal

I see only benefits with this project, plus the fact we already
have Golang projects doing their own code to communicate with
QEMU and I do believe we will make their lives easier.

> > It is also reasonable to say that libvirt would be better off
> > if it could auto-generate a client API for QEMU too, instead
> > of writing it by hand from a human reading the QAPI
> >
> > With regards,
> > Daniel

Cheers,
Victor

signature.asc
Description: PGP signature