[Qemu-commits] [qemu/qemu] dcba65: monitor: Add Monitor parameter to monitor_set_cpu()

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: dcba65f824817596e817a43f83ef83bac9099e76
      
https://github.com/qemu/qemu/commit/dcba65f824817596e817a43f83ef83bac9099e76
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M include/monitor/monitor.h
    M monitor/hmp-cmds.c
    M monitor/misc.c

  Log Message:
  -----------
  monitor: Add Monitor parameter to monitor_set_cpu()

Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_set_cpu().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201005155855.256490-2-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 87e6f4a4d6885006931b371771e2933c40700427
      
https://github.com/qemu/qemu/commit/87e6f4a4d6885006931b371771e2933c40700427
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M hw/core/machine-hmp-cmds.c
    M include/monitor/monitor.h
    M monitor/hmp-cmds.c
    M monitor/misc.c
    M softmmu/cpus.c

  Log Message:
  -----------
  monitor: Add Monitor parameter to monitor_get_cpu_index()

Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_get_cpu_index().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-3-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 947e47448dcc4e4d7a8b7c42b43acb3435b3ad35
      
https://github.com/qemu/qemu/commit/947e47448dcc4e4d7a8b7c42b43acb3435b3ad35
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M audio/wavcapture.c
    M dump/dump.c
    M hw/scsi/vhost-scsi.c
    M hw/virtio/vhost-vsock.c
    M include/monitor/monitor.h
    M migration/fd.c
    M monitor/hmp.c
    M monitor/misc.c
    M monitor/monitor.c
    M monitor/qmp-cmds-control.c
    M monitor/qmp-cmds.c
    M monitor/qmp.c
    M net/socket.c
    M net/tap.c
    M softmmu/cpus.c
    M stubs/monitor-core.c
    M tests/test-util-sockets.c
    M trace/control.c
    M util/qemu-error.c
    M util/qemu-print.c
    M util/qemu-sockets.c

  Log Message:
  -----------
  monitor: Use getter/setter functions for cur_mon

cur_mon really needs to be coroutine-local as soon as we move monitor
command handlers to coroutines and let them yield. As a first step, just
remove all direct accesses to cur_mon so that we can implement this in
the getter function later.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-4-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: ff04108a0e36e822519c517bd3bddbc1c7747c18
      
https://github.com/qemu/qemu/commit/ff04108a0e36e822519c517bd3bddbc1c7747c18
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M monitor/hmp.c
    M monitor/misc.c

  Log Message:
  -----------
  hmp: Update current monitor only in handle_hmp_command()

The current monitor is updated relatively early in the command handling
code even though only the command handler actually needs it.

The current monitor will become coroutine-local later, so we can only
update it when we know in which coroutine the command will be exectued.
Move it to handle_hmp_command() where this information will be
available.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201005155855.256490-5-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 57d3635e42752289388b0a178fd636acbf3b90fa
      
https://github.com/qemu/qemu/commit/57d3635e42752289388b0a178fd636acbf3b90fa
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M monitor/qmp.c

  Log Message:
  -----------
  qmp: Assert that no other monitor is active

monitor_qmp_dispatch() is never supposed to be called in the context of
another monitor, so assert that monitor_cur() is NULL instead of saving
and restoring it.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-6-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 41725fa7eda1d97576fc8c79b58d04a61629f40e
      
https://github.com/qemu/qemu/commit/41725fa7eda1d97576fc8c79b58d04a61629f40e
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M include/qapi/qmp/dispatch.h
    M monitor/qmp.c
    M qapi/qmp-dispatch.c
    M qga/main.c
    M stubs/monitor-core.c
    M tests/test-qmp-cmds.c

  Log Message:
  -----------
  qmp: Call monitor_set_cur() only in qmp_dispatch()

The correct way to set the current monitor for a coroutine handler will
be different than for a blocking handler, so monitor_set_cur() needs to
be called in qmp_dispatch().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-7-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: e69ee454b5f9dff3af48bcfc3d9691b3edb02fe2
      
https://github.com/qemu/qemu/commit/e69ee454b5f9dff3af48bcfc3d9691b3edb02fe2
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M include/monitor/monitor.h
    M monitor/hmp.c
    M monitor/monitor.c
    M qapi/qmp-dispatch.c
    M stubs/monitor-core.c

  Log Message:
  -----------
  monitor: Make current monitor a per-coroutine property

This way, a monitor command handler will still be able to access the
current monitor, but when it yields, all other code code will correctly
get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current
monitor of that coroutine.  Outside of coroutine context, we associate
the current monitor with the leader coroutine of the current thread.

Approaches to implement some form of coroutine local storage directly in
the coroutine core code have been considered and discarded because they
didn't end up being much more generic than the hash table and their
performance impact on coroutines not using coroutine local storage was
unclear. As the block layer uses a coroutine per I/O request, this is a
fast path and we have to be careful. It's safest to just stay out of
this path with code only used by the monitor.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201005155855.256490-8-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 04f22362f14b028c2632ce01e74e6a78c2b45e89
      
https://github.com/qemu/qemu/commit/04f22362f14b028c2632ce01e74e6a78c2b45e89
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M docs/devel/qapi-code-gen.txt
    M docs/sphinx/qapidoc.py
    M include/qapi/qmp/dispatch.h
    M scripts/qapi/commands.py
    M scripts/qapi/expr.py
    M scripts/qapi/introspect.py
    M scripts/qapi/schema.py
    M tests/qapi-schema/meson.build
    A tests/qapi-schema/oob-coroutine.err
    A tests/qapi-schema/oob-coroutine.json
    A tests/qapi-schema/oob-coroutine.out
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out
    M tests/qapi-schema/test-qapi.py
    M tests/test-qmp-cmds.c

  Log Message:
  -----------
  qapi: Add a 'coroutine' flag for commands

This patch adds a new 'coroutine' flag to QMP command definitions that
tells the QMP dispatcher that the command handler is safe to be run in a
coroutine.

The documentation of the new flag pretends that this flag is already
used as intended, which it isn't yet after this patch. We'll implement
this in another patch in this series.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-9-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 9ce44e2ce267caf5559904a201aa1986b0a8326b
      
https://github.com/qemu/qemu/commit/9ce44e2ce267caf5559904a201aa1986b0a8326b
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M include/qapi/qmp/dispatch.h
    M monitor/monitor-internal.h
    M monitor/monitor.c
    M monitor/qmp.c
    M qapi/qmp-dispatch.c
    M qapi/qmp-registry.c
    M util/aio-posix.c

  Log Message:
  -----------
  qmp: Move dispatcher to a coroutine

This moves the QMP dispatcher to a coroutine and runs all QMP command
handlers that declare 'coroutine': true in coroutine context so they
can avoid blocking the main loop while doing I/O or waiting for other
events.

For commands that are not declared safe to run in a coroutine, the
dispatcher drops out of coroutine context by calling the QMP command
handler from a bottom half.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20201005155855.256490-10-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: bb4b9ead95c3aaca84823e28dd9f11ccaa875c14
      
https://github.com/qemu/qemu/commit/bb4b9ead95c3aaca84823e28dd9f11ccaa875c14
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M docs/devel/qapi-code-gen.txt
    M monitor/hmp.c
    M monitor/monitor-internal.h

  Log Message:
  -----------
  hmp: Add support for coroutine command handlers

Often, QMP command handlers are not only called to handle QMP commands,
but also from a corresponding HMP command handler. In order to give them
a consistent environment, optionally run HMP command handlers in a
coroutine, too.

The implementation is a lot simpler than in QMP because for HMP, we
still block the VM while the coroutine is running.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201005155855.256490-11-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 26b0b698c00bd9176f86c539aeb680481fa19473
      
https://github.com/qemu/qemu/commit/26b0b698c00bd9176f86c539aeb680481fa19473
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M include/block/aio.h
    M util/async.c

  Log Message:
  -----------
  util/async: Add aio_co_reschedule_self()

Add a function that can be used to move the currently running coroutine
to a different AioContext (and therefore potentially a different
thread).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201005155855.256490-12-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: e336fd4c4b2fa04e5d6c7f8ee524bfd2d9e9e8f1
      
https://github.com/qemu/qemu/commit/e336fd4c4b2fa04e5d6c7f8ee524bfd2d9e9e8f1
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M block.c
    M include/block/block.h

  Log Message:
  -----------
  block: Add bdrv_co_enter()/leave()

Add a pair of functions to temporarily move the current coroutine to the
AioContext of a given BlockDriverState.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-13-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: 18c6ac1c6eb7cc541249585836659d0d3ed3a539
      
https://github.com/qemu/qemu/commit/18c6ac1c6eb7cc541249585836659d0d3ed3a539
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M block.c
    M include/block/block.h

  Log Message:
  -----------
  block: Add bdrv_lock()/unlock()

Inside of coroutine context, we can't directly use aio_context_acquire()
for the AioContext of a block node because we already own the lock of
the current AioContext and we need to avoid double locking to prevent
deadlocks.

This provides helper functions to lock the AioContext of a node only if
it's not the same as the current AioContext.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-14-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: eb94b81a94bce112e6b206df846c1551aaf6cab6
      
https://github.com/qemu/qemu/commit/eb94b81a94bce112e6b206df846c1551aaf6cab6
  Author: Kevin Wolf <kwolf@redhat.com>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M blockdev.c
    M hmp-commands.hx
    M qapi/block-core.json

  Log Message:
  -----------
  block: Convert 'block_resize' to coroutine

block_resize performs some I/O that could potentially take quite some
time, so use it as an example for the new 'coroutine': true annotation
in the QAPI schema.

bdrv_truncate() requires that we're already in the right AioContext for
the BlockDriverState if called in coroutine context. So instead of just
taking the AioContext lock, move the QMP handler coroutine to the
context.

Call blk_unref() only after switching back because blk_unref() may only
be called in the main thread.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201005155855.256490-15-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>


  Commit: b7092cda1b36ce687e65ab1831346f9529b781b8
      
https://github.com/qemu/qemu/commit/b7092cda1b36ce687e65ab1831346f9529b781b8
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-10-09 (Fri, 09 Oct 2020)

  Changed paths:
    M audio/wavcapture.c
    M block.c
    M blockdev.c
    M docs/devel/qapi-code-gen.txt
    M docs/sphinx/qapidoc.py
    M dump/dump.c
    M hmp-commands.hx
    M hw/core/machine-hmp-cmds.c
    M hw/scsi/vhost-scsi.c
    M hw/virtio/vhost-vsock.c
    M include/block/aio.h
    M include/block/block.h
    M include/monitor/monitor.h
    M include/qapi/qmp/dispatch.h
    M migration/fd.c
    M monitor/hmp-cmds.c
    M monitor/hmp.c
    M monitor/misc.c
    M monitor/monitor-internal.h
    M monitor/monitor.c
    M monitor/qmp-cmds-control.c
    M monitor/qmp-cmds.c
    M monitor/qmp.c
    M net/socket.c
    M net/tap.c
    M qapi/block-core.json
    M qapi/qmp-dispatch.c
    M qapi/qmp-registry.c
    M qga/main.c
    M scripts/qapi/commands.py
    M scripts/qapi/expr.py
    M scripts/qapi/introspect.py
    M scripts/qapi/schema.py
    M softmmu/cpus.c
    M stubs/monitor-core.c
    M tests/qapi-schema/meson.build
    A tests/qapi-schema/oob-coroutine.err
    A tests/qapi-schema/oob-coroutine.json
    A tests/qapi-schema/oob-coroutine.out
    M tests/qapi-schema/qapi-schema-test.json
    M tests/qapi-schema/qapi-schema-test.out
    M tests/qapi-schema/test-qapi.py
    M tests/test-qmp-cmds.c
    M tests/test-util-sockets.c
    M trace/control.c
    M util/aio-posix.c
    M util/async.c
    M util/qemu-error.c
    M util/qemu-print.c
    M util/qemu-sockets.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/armbru/tags/pull-monitor-2020-10-09' 
into staging

Monitor patches for 2020-10-09

# gpg: Signature made Fri 09 Oct 2020 06:16:51 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-monitor-2020-10-09:
  block: Convert 'block_resize' to coroutine
  block: Add bdrv_lock()/unlock()
  block: Add bdrv_co_enter()/leave()
  util/async: Add aio_co_reschedule_self()
  hmp: Add support for coroutine command handlers
  qmp: Move dispatcher to a coroutine
  qapi: Add a 'coroutine' flag for commands
  monitor: Make current monitor a per-coroutine property
  qmp: Call monitor_set_cur() only in qmp_dispatch()
  qmp: Assert that no other monitor is active
  hmp: Update current monitor only in handle_hmp_command()
  monitor: Use getter/setter functions for cur_mon
  monitor: Add Monitor parameter to monitor_get_cpu_index()
  monitor: Add Monitor parameter to monitor_set_cpu()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/497d415d76b9...b7092cda1b36