Re: [PATCH v3 0/4] qapi/migration: Dedup migration parameter objects and fix tls-authz crash

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Mon, Oct 16, 2023 at 09:08:40AM +0200, Markus Armbruster wrote:
>> Let me try to summarize our findings so far.
>
> Thanks.  I'll reply here instead of all the rest places.
>
>> 
>> PATCH 1 has been merged.  PATCH 2 has been queued, but not merged (not
>> sure why, accident?).
>
> (I don't know, either; could be in the next pull)
>
>> 
>> The remaining two are the actual de-triplication:
>> 
>> PATCH 3: Fuse MigrationParameters and MigrateSetParameters
>> 
>> PATCH 4: De-document MigrationParameter
>> 
>> The latter is a much simpler problem, so let's discuss it first.
>> 
>> 
>> Enum MigrationParameter is used only internally.  It's in the QAPI
>> schema just because we want code generated for it.  It shouldn't be
>> documented in the QEMU QMP Reference Manual, but is, because the
>> generator is too stupid to elide internal-only stuff.
>> 
>> PATCH 4 moves it out of the schema.  It has to add back the lost
>> generated code in hand-written form, which is a bit unfortunate.  I
>> proposed to instead drop most of the useless doc comment by exploiting a
>> QAPI generator loophole.
>> 
>> Aside: the QAPI generator should elide internal-only stuff from the QEMU
>> QMP Reference manual, and it should not require doc comments then.
>> Future work, let's not worry about it now.
>
> Just to double check: @MigrationParameter will not be exported in any form
> even today, including query-qmp-schema, am I right?

You are right.

Checking whether something is in the output of query-qmp-schema is easy:
look for it in the generated qapi-introspect.c.  Command names appear
like

    QLIT_QDICT(((QLitDictEntry[]) {
        [...]
        { "meta-type", QLIT_QSTR("command"), },
        { "name", QLIT_QSTR("query-migrate"), },
        [...]
    })),

Events names appear like

    QLIT_QDICT(((QLitDictEntry[]) {
        [...]
        { "meta-type", QLIT_QSTR("event"), },
        { "name", QLIT_QSTR("MIGRATION"), },
        [...]
    })),

Type names appear in comments instead of code, like

    /* "145" = MigrationParameters */

MigrationParameter does not appear.

>> The fusing of MigrationParameters and MigrateSetParameters is kind of
>> stuck.  Several options, all with drawbacks or problems:
>> 
>> 1. Pick StrOrNull for the tls_FOO members
>> 
>>    This is what PATCH 3 does.  Blocked on the pre-existing class of
>>    crash bugs discussed in
>> 
>>     Subject: QAPI string visitors crashes
>>     Message-ID: <875y3epv3y.fsf@pond.sub.org>
>>     875y3epv3y.fsf@pond.sub.org/">https://lore.kernel.org/qemu-devel/875y3epv3y.fsf@pond.sub.org/
>> 
>>    Needs fixing, but when a fix will be available is unclear.
>> 
>> 2. Pick str for the tls_FOO members
>> 
>>    This is what v1 did.  Incompatible change: JSON null no longer works.
>>    Libvirt doesn't use it (it uses deprecated "" instead), but we cannot
>>    know for sure nothing else out there uses it.
>> 
>>    I don't think reducing development friction (that's what
>>    de-duplication accomplishes) justifies breaking our compatibility
>>    promise.
>> 
>>    To keep the promise, we'd have to deprecate null, un-deprecate "",
>>    let the grace period pass, and only then de-duplicate.
>
> Is "" deprecated already anywhere?

Deprecation was cleary intended (see commit message of 01fa5598269), but
it looks like it wasn't (and isn't) properly documented.  We were much
less organized about deprecating stuff back then.

>> 3. Do nothing, live with the duplication
>> 
>>    Giving up like this would be sad.  Unless we commit to a more
>>    complete overhaul of migration's QAPI/QMP configuration interface,
>>    but I doubt we're ready for that.
>> 
>> Thoughts?
>
> I already went 3) on the patch I posted for avail-switchover-bw.  I don't
> know what's the best for 1) and 2), but if we can at least reduce
> duplication from 3->2 that's a progress.  I replied in the other thread for
> that loophole experiment.
>
> How hard it is to mark an object not requiring documentation on each of its
> field, if that's what we want in this case?  Currently the loophole didn't
> work for me for some reason.  If we can have a marker for objects to escape
> doc check legally, we can apply that to both @MigrationParameter and one
> other (perhaps @MigrationSetParameters).

I can see two useful QAPI generator features:

* Improved handling of missing member documentation

  Problem: many members lack documentation.  We silently generate
  documentation like

      name-of-member
          Not documented

  for them.

  Possible improvement: make missing member documentation a hard error,
  create a knob to suppress the error for a type.  Open question: how to
  best document member documentation is incomplete.

* Suppress documentation for internal-only definitions

  Problem: generated documentation covers everything, even types that
  aren't visible in QMP.  The irrelevant material is distracting and
  possibly confusing for users, and may be bothersome to maintain for
  developers.

  Possible improvement: include only the types visible in QMP in
  documentation, similar to how we do for query-qmp-schema.  Open
  question: what level of documentation to require for internal-only
  types.