[Qemu-commits] [qemu/qemu] 9fe7ef: migration: Drop unused VMSTATE_FLOAT64 support

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 9fe7ef8b6646054ab040a2bc765d0cb43a993cd8
      
https://github.com/qemu/qemu/commit/9fe7ef8b6646054ab040a2bc765d0cb43a993cd8
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M include/migration/vmstate.h
    M migration/vmstate-types.c

  Log Message:
  -----------
  migration: Drop unused VMSTATE_FLOAT64 support

Commit ef96e3ae9698d6 in January 2019 removed the last user of the
VMSTATE_FLOAT64* macros. These were used by targets which defined
their floating point register file as an array of 'float64'.

We used to try to maintain a stricter distinction between
'float64' (a type for holding an integer representing an IEEE float)
and 'uint64_t', including having a debug option for 'float64' being
a struct and supposedly mandatory macros for converting between
float64 and uint64_t. We no longer think that's a usefully
strong distinction to draw and we allow ourselves to freely
assume that float64 really is just a 64-bit integer type, so
for new targets we would simply recommend use of the uint64_t type
for a floating point register file. The float64 type remains
as a useful way of documenting in the type signature of helper
functions and the like that they expect to receive an IEEE float
from the TCG generated code rather than an arbitrary integer.

Since the VMSTATE_FLOAT64* macros have no remaining users and
we don't recommend new code uses them, delete them.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20201022120830.5938-1-peter.maydell@linaro.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 01371c5821c432697a636d9c3be2ba27685562a8
      
https://github.com/qemu/qemu/commit/01371c5821c432697a636d9c3be2ba27685562a8
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/block.c
    M migration/rdma.c

  Log Message:
  -----------
  migration: Do not use C99 // comments

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-2-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 29fccade10ac38f2a7aba548eb04d7396a287bb0
      
https://github.com/qemu/qemu/commit/29fccade10ac38f2a7aba548eb04d7396a287bb0
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/block.c
    M migration/ram.c

  Log Message:
  -----------
  migration: Don't use '#' flag of printf format

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-3-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 395cb450095408e0f7b57d69c22a14120f47dc52
      
https://github.com/qemu/qemu/commit/395cb450095408e0f7b57d69c22a14120f47dc52
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/migration.c
    M migration/postcopy-ram.c
    M migration/ram.c
    M migration/savevm.c
    M migration/vmstate.c

  Log Message:
  -----------
  migration: Add spaces around operator

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-4-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: f16aee44b46ece5df0661866b0cacd9e0e52f408
      
https://github.com/qemu/qemu/commit/f16aee44b46ece5df0661866b0cacd9e0e52f408
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/migration.h

  Log Message:
  -----------
  migration: Open brace '{' following struct go on the same line

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-5-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: f4c51a6bfdc0469f34cf53c803e15fc26f87238c
      
https://github.com/qemu/qemu/commit/f4c51a6bfdc0469f34cf53c803e15fc26f87238c
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  migration: Add braces {} for if statement

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-6-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 49324e939c9d9c963397978ea326665e8a5cf4b5
      
https://github.com/qemu/qemu/commit/49324e939c9d9c963397978ea326665e8a5cf4b5
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/ram.c
    M migration/savevm.c

  Log Message:
  -----------
  migration: Do not initialise statics and globals to 0 or NULL

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-7-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: cbfc71b52b067b3b70a7ace45ff223b58d63a540
      
https://github.com/qemu/qemu/commit/cbfc71b52b067b3b70a7ace45ff223b58d63a540
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/rdma.c

  Log Message:
  -----------
  migration: Open brace '{' following function declarations go on the next line

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-8-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 0bcae62333e67a7d8831cab97a3fbf5ea5498cd3
      
https://github.com/qemu/qemu/commit/0bcae62333e67a7d8831cab97a3fbf5ea5498cd3
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/rdma.c

  Log Message:
  -----------
  migration: Delete redundant spaces

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-9-git-send-email-yubihong@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: fe80c0241dc493a66a0805fe016ed28505492fed
      
https://github.com/qemu/qemu/commit/fe80c0241dc493a66a0805fe016ed28505492fed
  Author: Bihong Yu <yubihong@huawei.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/block.c
    M migration/page_cache.c
    M migration/trace-events

  Log Message:
  -----------
  migration: using trace_ to replace DPRINTF

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Message-Id: <1603179176-5360-1-git-send-email-yubihong@huawei.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: eef621c4e617774ea894dfe303eb89b8a692aa2e
      
https://github.com/qemu/qemu/commit/eef621c4e617774ea894dfe303eb89b8a692aa2e
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/postcopy-ram.c

  Log Message:
  -----------
  migration: Pass incoming state into qemu_ufd_copy_ioctl()

It'll be used in follow up patches to access more fields out of it.  Meanwhile
fetch the userfaultfd inside the function.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-2-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 7a267fc49b5a06cb395fe437996d5df517783ff8
      
https://github.com/qemu/qemu/commit/7a267fc49b5a06cb395fe437996d5df517783ff8
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/migration.c
    M migration/migration.h

  Log Message:
  -----------
  migration: Introduce migrate_send_rp_message_req_pages()

This is another layer wrapper for sending a page request to the source VM.  The
new migrate_send_rp_message_req_pages() will be used elsewhere in coming
patches.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-3-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 8f8bfffcf1b486cee9a3bc79bb9b174682b06e22
      
https://github.com/qemu/qemu/commit/8f8bfffcf1b486cee9a3bc79bb9b174682b06e22
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/migration.c
    M migration/migration.h
    M migration/postcopy-ram.c
    M migration/trace-events

  Log Message:
  -----------
  migration: Maintain postcopy faulted addresses

Maintain a list of faulted addresses on the destination host for which we're
waiting on.  This is implemented using a GTree rather than a real list to make
sure even there're plenty of vCPUs/threads that are faulting, the lookup will
still be fast with O(log(N)) (because we'll do that after placing each page).
It should bring a slight overhead, but ideally that shouldn't be a big problem
simply because in most cases the requested page list will be short.

Actually we did similar things for postcopy blocktime measurements.  This patch
didn't use that simply because:

  (1) blocktime measurement is towards vcpu threads only, but here we need to
      record all faulted addresses, including main thread and external
      thread (like, DPDK via vhost-user).

  (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we
      don't want to add that extra dependency on the kernel version since not
      necessary.  E.g., we don't need to know which thread faulted on which
      page, we also don't care about multiple threads faulting on the same
      page.  But we only care about what addresses are faulted so waiting for a
      page copying from src.

  (3) blocktime measurement is not enabled by default.  However we need this by
      default especially for postcopy recover.

Another thing to mention is that this patch introduced a new mutex to serialize
the receivedmap and the page_requested tree, however that serialization does
not cover other procedures like UFFDIO_COPY.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-4-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: 0c26781c0937324d175b8105bc96ccce778d9760
      
https://github.com/qemu/qemu/commit/0c26781c0937324d175b8105bc96ccce778d9760
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/savevm.c
    M migration/trace-events

  Log Message:
  -----------
  migration: Sync requested pages after postcopy recovery

We synchronize the requested pages right after a postcopy recovery happens.
This helps to synchronize the prioritized pages on source so that the faulted
threads can be served faster.

Reported-by: Xiaohui Li <xiaohli@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-5-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: d246ea5039fd6d5344becd0943fcbb7f8e6bbfe7
      
https://github.com/qemu/qemu/commit/d246ea5039fd6d5344becd0943fcbb7f8e6bbfe7
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M migration/migration.c

  Log Message:
  -----------
  migration/postcopy: Release fd before going into 'postcopy-pause'

Logically below race could trigger with the old code:

          test program                        migration thread
          ------------                        ----------------
       wait_until('postcopy-pause')
                                          postcopy_pause()
                                            set_state('postcopy-pause')
       do_postcopy_recover()
         arm s->to_dst_file with new fd
                                            release s->to_dst_file [1]

Here [1] could have released the just-installed recoverying channel.  Then the
migration could hang without really resuming.

Instead, it should be very safe to release the fd before setting the state into
'postcopy-pause', because there's no reason for any other thread to touch it
during 'postcopy-active'.

Dave reported a very rare postcopy recovery hang that the migration-test
program waited for the migration to complete in migrate_postcopy_complete().
We do suspect it's the same thing that we're gonna fix here.  Hard to tell.
However since we've noticed this, fix this irrelevant of the hang report.

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-6-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: a47295014de56e108f359ec859d5499b851f62b8
      
https://github.com/qemu/qemu/commit/a47295014de56e108f359ec859d5499b851f62b8
  Author: Peter Xu <peterx@redhat.com>
  Date:   2020-10-26 (Mon, 26 Oct 2020)

  Changed paths:
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  migration-test: Only hide error if !QTEST_LOG

The errors are very useful when debugging qtest failures, especially when
QTEST_LOG=1 is set.  Let's allow override MigrateStart.hide_stderr when
QTEST_LOG=1 is specified, because that means the user wants to be verbose.

Not very nice to introduce the first QTEST_LOG env access in migration-test.c,
however it should be handy.  Without this patch, I was hacking error_report()
when debugging such errors.  Let's make things easier.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-7-peterx@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>


  Commit: d55450df995d6223486db11c66491cbf6c131523
      
https://github.com/qemu/qemu/commit/d55450df995d6223486db11c66491cbf6c131523
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-10-27 (Tue, 27 Oct 2020)

  Changed paths:
    M include/migration/vmstate.h
    M migration/block.c
    M migration/migration.c
    M migration/migration.h
    M migration/page_cache.c
    M migration/postcopy-ram.c
    M migration/ram.c
    M migration/rdma.c
    M migration/savevm.c
    M migration/trace-events
    M migration/vmstate-types.c
    M migration/vmstate.c
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20201026a' 
into staging

migration pull: 2020-10-26

Another go at Peter's postcopy fixes

Cleanups from Bihong Yu and Peter Maydell.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

# gpg: Signature made Mon 26 Oct 2020 16:17:03 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" 
[full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20201026a:
  migration-test: Only hide error if !QTEST_LOG
  migration/postcopy: Release fd before going into 'postcopy-pause'
  migration: Sync requested pages after postcopy recovery
  migration: Maintain postcopy faulted addresses
  migration: Introduce migrate_send_rp_message_req_pages()
  migration: Pass incoming state into qemu_ufd_copy_ioctl()
  migration: using trace_ to replace DPRINTF
  migration: Delete redundant spaces
  migration: Open brace '{' following function declarations go on the next line
  migration: Do not initialise statics and globals to 0 or NULL
  migration: Add braces {} for if statement
  migration: Open brace '{' following struct go on the same line
  migration: Add spaces around operator
  migration: Don't use '#' flag of printf format
  migration: Do not use C99 // comments
  migration: Drop unused VMSTATE_FLOAT64 support

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/091e3e3dbc49...d55450df995d