[Qemu-commits] [qemu/qemu] a53e37: spapr_pci: Fix memory leak of vmstate_spapr_pci

[Peter Maydell]

  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: a53e37d12b807301cc3cb611b345bab5146fa4a5
      
https://github.com/qemu/qemu/commit/a53e37d12b807301cc3cb611b345bab5146fa4a5
  Author: Jinhao Gao <gaojinhao@huawei.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M hw/ppc/spapr_pci.c

  Log Message:
  -----------
  spapr_pci: Fix memory leak of vmstate_spapr_pci

When VM migrate VMState of spapr_pci, the field(msi_devs) of spapr_pci
having a flag of VMS_ALLOC need to allocate memory. If the src doesn't free
memory of msi_devs in SaveStateEntry of spapr_pci after QEMUFile save
VMState of spapr_pci, it may result in memory leak of msi_devs. We add the
post_save func to free memory, which prevents memory leak.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Jinhao Gao <gaojinhao@huawei.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20201231061020.828-2-gaojinhao@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 3dd03994d9c80959db1debc3dfc9b5b14e708065
      
https://github.com/qemu/qemu/commit/3dd03994d9c80959db1debc3dfc9b5b14e708065
  Author: Jinhao Gao <gaojinhao@huawei.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/savevm.c

  Log Message:
  -----------
  savevm: Fix memory leak of vmstate_configuration

When VM migrate VMState of configuration, the fields(name and capabilities)
of configuration having a flag of VMS_ALLOC need to allocate memory. If the
src doesn't free memory of capabilities in SaveState after save VMState of
configuration, or the dst doesn't free memory of name and capabilities in post
load of configuration, it may result in memory leak of name and capabilities.
We free memory in configuration_post_save and configuration_post_load func,
which prevents memory leak.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Jinhao Gao <gaojinhao@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20201231061020.828-3-gaojinhao@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 7c2c73ec60ca710000497a982abdf2b913316ec7
      
https://github.com/qemu/qemu/commit/7c2c73ec60ca710000497a982abdf2b913316ec7
  Author: Jinhao Gao <gaojinhao@huawei.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/vmstate.c

  Log Message:
  -----------
  vmstate: Fix memory leak in vmstate_handle_alloc()

Some memory allocated for fields having a flag of VMS_ALLOC in SaveState
may not free before VM load vmsd in migration. So we pre-free memory before
allocation in vmstate_handle_alloc() to avoid memleaks.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Jinhao Gao <gaojinhao@huawei.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20201231061020.828-4-gaojinhao@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: b2fac9ddd366f77df52d085d0fb0e17acafda4a8
      
https://github.com/qemu/qemu/commit/b2fac9ddd366f77df52d085d0fb0e17acafda4a8
  Author: Wainer dos Santos Moschetta <wainersm@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/qemu-file.c

  Log Message:
  -----------
  migration/qemu-file: Fix maybe uninitialized on qemu_get_buffer_in_place()

Fixed error when compiling migration/qemu-file.c with 
-Werror=maybe-uninitialized
as shown here:

../migration/qemu-file.c: In function 'qemu_get_buffer_in_place':
../migration/qemu-file.c:604:18: error: 'src' may be used uninitialized in this 
function [-Werror=maybe-uninitialized]
  604 |             *buf = src;
      |             ~~~~~^~~~~
cc1: all warnings being treated as errors

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210128130625.569900-1-wainersm@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: b0d792ac3af142241ccadd819e995ef32d396f56
      
https://github.com/qemu/qemu/commit/b0d792ac3af142241ccadd819e995ef32d396f56
  Author: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M migration/ram.c
    M migration/ram.h
    M qapi/migration.json

  Log Message:
  -----------
  migration: introduce 'background-snapshot' migration capability

Add new capability to 'qapi/migration.json' schema.
Update migrate_caps_check() to validate enabled capability set
against introduced one. Perform checks for required kernel features
and compatibility with guest memory backends.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210129101407.103458-2-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 1f292091ca8454bb1effa02f3317c6424cf55530
      
https://github.com/qemu/qemu/commit/1f292091ca8454bb1effa02f3317c6424cf55530
  Author: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/exec/memory.h
    A include/qemu/userfaultfd.h
    M util/meson.build
    M util/trace-events
    A util/userfaultfd.c

  Log Message:
  -----------
  migration: introduce UFFD-WP low-level interface helpers

Glue code to the userfaultfd kernel implementation.
Querying feature support, createing file descriptor, feature control,
memory region registration, IOCTLs on registered registered regions.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210129101407.103458-3-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Fixed up range.start casting for 32bit


  Commit: 8fad5d76e97872db1f3ca4996ab5a5ff1ed8d69c
      
https://github.com/qemu/qemu/commit/8fad5d76e97872db1f3ca4996ab5a5ff1ed8d69c
  Author: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/exec/memory.h
    M migration/ram.c
    M migration/ram.h
    M migration/trace-events

  Log Message:
  -----------
  migration: support UFFD write fault processing in ram_save_iterate()

In this particular implementation the same single migration
thread is responsible for both normal linear dirty page
migration and procesing UFFD page fault events.

Processing write faults includes reading UFFD file descriptor,
finding respective RAM block and saving faulting page to
the migration stream. After page has been saved, write protection
can be removed. Since asynchronous version of qemu_put_buffer()
is expected to be used to save pages, we also have to flush
migraion stream prior to un-protecting saved memory range.

Write protection is being removed for any previously protected
memory chunk that has hit the migration stream. That's valid
for pages from linear page scan along with write fault pages.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210129101407.103458-4-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  fixup pagefault.address cast for 32bit


  Commit: a2208f3a400ed315f2c94330e6582d32f62bff96
      
https://github.com/qemu/qemu/commit/a2208f3a400ed315f2c94330e6582d32f62bff96
  Author: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M migration/savevm.c
    M migration/savevm.h

  Log Message:
  -----------
  migration: implementation of background snapshot thread

Introducing implementation of 'background' snapshot thread
which in overall follows the logic of precopy migration
while internally utilizes completely different mechanism
to 'freeze' vmstate at the start of snapshot creation.

This mechanism is based on userfault_fd with wr-protection
support and is Linux-specific.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210129101407.103458-5-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 88d41d9ff2c668cf1ff78adfac9a557239d85127
      
https://github.com/qemu/qemu/commit/88d41d9ff2c668cf1ff78adfac9a557239d85127
  Author: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    A scripts/userfaultfd-wrlat.py

  Log Message:
  -----------
  migration: introduce 'userfaultfd-wrlat.py' script

Add BCC/eBPF script to analyze userfaultfd write fault latency distribution.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210129101407.103458-6-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 48966420cc3d776dadd11b8843130dab6e9a429b
      
https://github.com/qemu/qemu/commit/48966420cc3d776dadd11b8843130dab6e9a429b
  Author: Markus Armbruster <armbru@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M monitor/hmp-cmds.c
    M qapi/migration.json

  Log Message:
  -----------
  migration: Fix migrate-set-parameters argument validation

Commit 741d4086c8 "migration: Use proper types in json" (v2.12.0)
switched MigrationParameters to narrower integer types, and removed
the simplified qmp_migrate_set_parameters()'s argument checking
accordingly.

Good idea, except qmp_migrate_set_parameters() takes
MigrateSetParameters, not MigrationParameters.  Its job is updating
migrate_get_current()->parameters (which *is* of type
MigrationParameters) according to its argument.  The integers now get
truncated silently.  Reproducer:

    ---> {'execute': 'query-migrate-parameters'}
    <--- {"return": {[...] "compress-threads": 8, [...]}}
    ---> {"execute": "migrate-set-parameters", "arguments": 
{"compress-threads": 257}}
    <--- {"return": {}}
    ---> {'execute': 'query-migrate-parameters'}
    <--- {"return": {[...] "compress-threads": 1, [...]}}

Fix by resynchronizing MigrateSetParameters with MigrationParameters.

Fixes: 741d4086c856320807a2575389d7c0505578270b
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202141734.2488076-2-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 2528ade9b7c324e8fcb8dfe7c701b85ef30aa832
      
https://github.com/qemu/qemu/commit/2528ade9b7c324e8fcb8dfe7c701b85ef30aa832
  Author: Markus Armbruster <armbru@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M migration/page_cache.c
    M migration/page_cache.h
    M migration/ram.c
    M migration/ram.h
    M qapi/migration.json

  Log Message:
  -----------
  migration: Clean up signed vs. unsigned XBZRLE cache-size

73af8dd8d7 "migration: Make xbzrle_cache_size a migration
parameter" (v2.11.0) made the new parameter unsigned (QAPI type
'size', uint64_t in C).  It neglected to update existing code, which
continues to use int64_t.

migrate_xbzrle_cache_size() returns the new parameter.  Adjust its
return type.

QMP query-migrate-cache-size returns migrate_xbzrle_cache_size().
Adjust its return type.

migrate-set-parameters passes the new parameter to
xbzrle_cache_resize().  Adjust its parameter type.

xbzrle_cache_resize() passes it on to cache_init().  Adjust its
parameter type.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202141734.2488076-3-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 2cd1fff9f2dbd8e121171b551fa892f0918eadb8
      
https://github.com/qemu/qemu/commit/2cd1fff9f2dbd8e121171b551fa892f0918eadb8
  Author: Markus Armbruster <armbru@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/page_cache.c

  Log Message:
  -----------
  migration: Fix cache_init()'s "Failed to allocate" error messages

cache_init() attempts to handle allocation failure.  The two error
messages are garbage, as untested error messages commonly are:

    Parameter 'cache size' expects Failed to allocate cache
    Parameter 'cache size' expects Failed to allocate page cache

Fix them to just

    Failed to allocate cache
    Failed to allocate page cache

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202141734.2488076-4-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: b785bfb9b3c43695dbd70e8c2a050d0ebc7a00f6
      
https://github.com/qemu/qemu/commit/b785bfb9b3c43695dbd70e8c2a050d0ebc7a00f6
  Author: Markus Armbruster <armbru@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/migration.c

  Log Message:
  -----------
  migration: Fix a few absurdly defective error messages

migrate_params_check() has a number of error messages of the form

    Parameter 'NAME' expects is invalid, it should be ...

Fix them to something like

    Parameter 'NAME' expects a ...

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202141734.2488076-5-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: db029de929a26a14efb2ce2eaaff8ce6662a9f49
      
https://github.com/qemu/qemu/commit/db029de929a26a14efb2ce2eaaff8ce6662a9f49
  Author: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/migration.c
    M migration/savevm.c
    M migration/savevm.h
    M qapi/migration.json

  Log Message:
  -----------
  migration: Add blocker information

Modify query-migrate so that it has a flag indicating if outbound
migration is blocked, and if it is a list of reasons.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202135522.127380-2-dgilbert@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: d861f71e5fb73ec7725b4b5b9d73404d1b4beb67
      
https://github.com/qemu/qemu/commit/d861f71e5fb73ec7725b4b5b9d73404d1b4beb67
  Author: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M monitor/hmp-cmds.c

  Log Message:
  -----------
  migration: Display the migration blockers

Update 'info migrate' to display migration blocking information.
If the outbound migration is not blocked, there is no change, however
if it is blocked a message is displayed with a list of reasons why,
e.g.

qemu-system-x86_64 -nographic  -smp 4 -m 4G -M pc,usb=on \
 -chardev null,id=n -device usb-serial,chardev=n \
 -virtfs local,path=/home,mount_tag=fs,security_model=none \
 -drive if=virtio,file=myimage.qcow2

(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
clear-bitmap-shift: 18
Outgoing migration blocked:
  Migration is disabled when VirtFS export path '/home' is mounted in the guest 
using mount_tag 'fs'
  non-migratable device: 0000:00:01.2/1/usb-serial

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202135522.127380-3-dgilbert@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 9cf0f52c9ac54c0cc3bd7d327b41bf6c145de602
      
https://github.com/qemu/qemu/commit/9cf0f52c9ac54c0cc3bd7d327b41bf6c145de602
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M block/monitor/block-hmp-cmds.c
    M block/snapshot.c
    M include/block/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c
    M replay/replay-debugging.c
    M tests/qemu-iotests/267.out

  Log Message:
  -----------
  block: push error reporting into bdrv_all_*_snapshot functions

The bdrv_all_*_snapshot functions return a BlockDriverState pointer
for the invalid backend, which the callers then use to report an
error message. In some cases multiple callers are reporting the
same error message, but with slightly different text. In the future
there will be more error scenarios for some of these methods, which
will benefit from fine grained error message reporting. So it is
helpful to push error reporting down a level.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[PMD: Initialize variables]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210204124834.774401-2-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: bc8202e26261806ab68372009790bd94a07a6426
      
https://github.com/qemu/qemu/commit/bc8202e26261806ab68372009790bd94a07a6426
  Author: Philippe Mathieu-Daudé <philmd@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/migration/snapshot.h
    M migration/savevm.c
    M replay/replay-debugging.c
    M replay/replay-snapshot.c

  Log Message:
  -----------
  migration: Make save_snapshot() return bool, not 0/-1

Just for consistency, following the example documented since
commit e3fe3988d7 ("error: Document Error API usage rules"),
return a boolean value indicating an error is set or not.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210204124834.774401-3-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 79605228f9da2c537090408f82776b465b247023
      
https://github.com/qemu/qemu/commit/79605228f9da2c537090408f82776b465b247023
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/migration/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c
    M replay/replay-snapshot.c
    M softmmu/vl.c

  Log Message:
  -----------
  migration: stop returning errno from load_snapshot()

None of the callers care about the errno value since there is a full
Error object populated. This gives consistency with save_snapshot()
which already just returns a boolean value.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[PMD: Return false/true instead of -1/0, document function]
Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210204124834.774401-4-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 17ec2a676906d1931e1b7424a54f8f8f74a71a48
      
https://github.com/qemu/qemu/commit/17ec2a676906d1931e1b7424a54f8f8f74a71a48
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M block/monitor/block-hmp-cmds.c
    M block/snapshot.c
    M include/block/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c
    M replay/replay-debugging.c

  Log Message:
  -----------
  block: add ability to specify list of blockdevs during snapshot

When running snapshot operations, there are various rules for which
blockdevs are included/excluded. While this provides reasonable default
behaviour, there are scenarios that are not well handled by the default
logic. Some of the conditions do not have a single correct answer.

Thus there needs to be a way for the mgmt app to provide an explicit
list of blockdevs to perform snapshots across. This can be achieved
by passing a list of node names that should be used.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-5-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 143e5b881b1af87627c50b760a27bac176c2197a
      
https://github.com/qemu/qemu/commit/143e5b881b1af87627c50b760a27bac176c2197a
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M block/monitor/block-hmp-cmds.c
    M block/snapshot.c
    M include/block/snapshot.h
    M migration/savevm.c
    M replay/replay-debugging.c
    M tests/qemu-iotests/267.out

  Log Message:
  -----------
  block: allow specifying name of block device for vmstate storage

Currently the vmstate will be stored in the first block device that
supports snapshots. Historically this would have usually been the
root device, but with UEFI it might be the variable store. There
needs to be a way to override the choice of block device to store
the state in.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-6-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: ca52586bdcd10e7a7026eaa8c657abe536555e60
      
https://github.com/qemu/qemu/commit/ca52586bdcd10e7a7026eaa8c657abe536555e60
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M block/monitor/block-hmp-cmds.c
    M block/snapshot.c
    M include/block/snapshot.h
    M migration/savevm.c
    M replay/replay-debugging.c

  Log Message:
  -----------
  block: rename and alter bdrv_all_find_snapshot semantics

Currently bdrv_all_find_snapshot() will return 0 if it finds
a snapshot, -1 if an error occurs, or if it fails to find a
snapshot. New callers to be added want to distinguish between
the error scenario and failing to find a snapshot.

Rename it to bdrv_all_has_snapshot and make it return -1 on
error, 0 if no snapshot is found and 1 if snapshot is found.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-7-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 854a1c4c03efc278b7629c055bc4b41a6a439ffc
      
https://github.com/qemu/qemu/commit/854a1c4c03efc278b7629c055bc4b41a6a439ffc
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/migration/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c
    M replay/replay-debugging.c
    M replay/replay-snapshot.c

  Log Message:
  -----------
  migration: control whether snapshots are ovewritten

The traditional HMP "savevm" command will overwrite an existing snapshot
if it already exists with the requested name. This new flag allows this
to be controlled allowing for safer behaviour with a future QMP command.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-8-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 50d73fdbefadfd47fde14fdbb5e837ee0f413d0e
      
https://github.com/qemu/qemu/commit/50d73fdbefadfd47fde14fdbb5e837ee0f413d0e
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/migration/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c
    M replay/replay-debugging.c
    M replay/replay-snapshot.c
    M softmmu/vl.c

  Log Message:
  -----------
  migration: wire up support for snapshot device selection

Modify load_snapshot/save_snapshot to accept the device list and vmstate
node name parameters previously added to the block layer.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-9-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: ef18022bc9a5b561aad68a1a2a3166eb6c321843
      
https://github.com/qemu/qemu/commit/ef18022bc9a5b561aad68a1a2a3166eb6c321843
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M include/migration/snapshot.h
    M migration/savevm.c
    M monitor/hmp-cmds.c

  Log Message:
  -----------
  migration: introduce a delete_snapshot wrapper

Make snapshot deletion consistent with the snapshot save
and load commands by using a wrapper around the blockdev
layer. The main difference is that we get upfront validation
of the passed in device list (if any).

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-10-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: f3e5c2d672489d769dae16f83501104fed3a79f3
      
https://github.com/qemu/qemu/commit/f3e5c2d672489d769dae16f83501104fed3a79f3
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M tests/qemu-iotests/common.qemu

  Log Message:
  -----------
  iotests: add support for capturing and matching QMP events

When using the _launch_qemu and _send_qemu_cmd functions from
common.qemu, any QMP events get mixed in with the output from
the commands and responses.

This makes it difficult to write a test case as the ordering
of events in the output is not stable.

This introduces a variable 'capture_events' which can be set
to a list of event names. Any events listed in this variable
will not be printed, instead collected in the $QEMU_EVENTS
environment variable.

A new '_wait_event' function can be invoked to collect events
at a fixed point in time. The function will first pull events
cached in $QEMU_EVENTS variable, and if none are found, will
then read more from QMP.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-11-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 8dec2b22208124ee302c39c397dbfe87b40c6756
      
https://github.com/qemu/qemu/commit/8dec2b22208124ee302c39c397dbfe87b40c6756
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  iotests: fix loading of common.config from tests/ subdir

common.rc assumes it is being sourced from the same directory and
so also tries to source common.config from the current working
directory. With the ability to now have named tests in the tests/
subdir we need to check two locations for common.config.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-12-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: ef74d46576a9e5aff96f285b74150f341a525688
      
https://github.com/qemu/qemu/commit/ef74d46576a9e5aff96f285b74150f341a525688
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M migration/savevm.c
    M qapi/job.json
    M qapi/migration.json

  Log Message:
  -----------
  migration: introduce snapshot-{save, load, delete} QMP commands

savevm, loadvm and delvm are some of the few HMP commands that have never
been converted to use QMP. The reasons for the lack of conversion are
that they blocked execution of the event thread, and the semantics
around choice of disks were ill-defined.

Despite this downside, however, libvirt and applications using libvirt
have used these commands for as long as QMP has existed, via the
"human-monitor-command" passthrough command. IOW, while it is clearly
desirable to be able to fix the problems, they are not a blocker to
all real world usage.

Meanwhile there is a need for other features which involve adding new
parameters to the commands. This is possible with HMP passthrough, but
it provides no reliable way for apps to introspect features, so using
QAPI modelling is highly desirable.

This patch thus introduces new snapshot-{load,save,delete} commands to
QMP that are intended to replace the old HMP counterparts. The new
commands are given different names, because they will be using the new
QEMU job framework and thus will have diverging behaviour from the HMP
originals. It would thus be misleading to keep the same name.

While this design uses the generic job framework, the current impl is
still blocking. The intention that the blocking problem is fixed later.
None the less applications using these new commands should assume that
they are asynchronous and thus wait for the job status change event to
indicate completion.

In addition to using the job framework, the new commands require the
caller to be explicit about all the block device nodes used in the
snapshot operations, with no built-in default heuristics in use.

Note that the existing "query-named-block-nodes" can be used to query
what snapshots currently exist for block nodes.

Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-13-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: removed tests for now, the output ordering isn't
deterministic


  Commit: b749a9213ac7ab03928272c10fef9089ea8662dd
      
https://github.com/qemu/qemu/commit/b749a9213ac7ab03928272c10fef9089ea8662dd
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2021-02-04 (Thu, 04 Feb 2021)

  Changed paths:
    M block/monitor/block-hmp-cmds.c
    M block/snapshot.c
    M hw/ppc/spapr_pci.c
    M include/block/snapshot.h
    M include/exec/memory.h
    M include/migration/snapshot.h
    A include/qemu/userfaultfd.h
    M migration/migration.c
    M migration/migration.h
    M migration/page_cache.c
    M migration/page_cache.h
    M migration/qemu-file.c
    M migration/ram.c
    M migration/ram.h
    M migration/savevm.c
    M migration/savevm.h
    M migration/trace-events
    M migration/vmstate.c
    M monitor/hmp-cmds.c
    M qapi/job.json
    M qapi/migration.json
    M replay/replay-debugging.c
    M replay/replay-snapshot.c
    A scripts/userfaultfd-wrlat.py
    M softmmu/vl.c
    M tests/qemu-iotests/267.out
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/common.rc
    M util/meson.build
    M util/trace-events
    A util/userfaultfd.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210204a' 
into staging

Migration pull 2020-02-04

 New snapshot features:
   a) Andrey's RAM snapshot feature using userfault-wp
   b) Dan's native-QMP snapshots

Cleanups:
   c) Jinhao's memory leeak fixes
   d) Wainer's maybe unitialized fix
   e) Markus's parameter fixes

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

# gpg: Signature made Thu 04 Feb 2021 16:38:15 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" 
[full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20210204a: (27 commits)
  migration: introduce snapshot-{save, load, delete} QMP commands
  iotests: fix loading of common.config from tests/ subdir
  iotests: add support for capturing and matching QMP events
  migration: introduce a delete_snapshot wrapper
  migration: wire up support for snapshot device selection
  migration: control whether snapshots are ovewritten
  block: rename and alter bdrv_all_find_snapshot semantics
  block: allow specifying name of block device for vmstate storage
  block: add ability to specify list of blockdevs during snapshot
  migration: stop returning errno from load_snapshot()
  migration: Make save_snapshot() return bool, not 0/-1
  block: push error reporting into bdrv_all_*_snapshot functions
  migration: Display the migration blockers
  migration: Add blocker information
  migration: Fix a few absurdly defective error messages
  migration: Fix cache_init()'s "Failed to allocate" error messages
  migration: Clean up signed vs. unsigned XBZRLE cache-size
  migration: Fix migrate-set-parameters argument validation
  migration: introduce 'userfaultfd-wrlat.py' script
  migration: implementation of background snapshot thread
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/72020a76d5c3...b749a9213ac7