[Qemu-commits] [qemu/qemu] 1bbbf3: block: Use QEMU_IS_ALIGNED

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 1bbbf32d5fffe334531c315d7bd865fdfb67b6c5
      
https://github.com/qemu/qemu/commit/1bbbf32d5fffe334531c315d7bd865fdfb67b6c5
  Author: Nir Soffer <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/bochs.c
    M block/cloop.c
    M block/dmg.c
    M block/io.c
    M block/qcow2-cluster.c
    M block/qcow2.c
    M block/vvfat.c
    M qemu-img.c

  Log Message:
  -----------
  block: Use QEMU_IS_ALIGNED

Replace instances of:

    (n & (BDRV_SECTOR_SIZE - 1)) == 0

And:

   (n & ~BDRV_SECTOR_MASK) == 0

With:

    QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE)

Which reveals the intent of the code better, and makes it easier to
locate the code checking alignment.

Signed-off-by: Nir Soffer <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 8972571509f15bd8e31fa9ded795a11e22c0d1d3
      
https://github.com/qemu/qemu/commit/8972571509f15bd8e31fa9ded795a11e22c0d1d3
  Author: Nir Soffer <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M include/block/block.h
    M migration/block.c

  Log Message:
  -----------
  block: Remove unused masks

Replace confusing usage:

    ~BDRV_SECTOR_MASK

With more clear:

    (BDRV_SECTOR_SIZE - 1)

Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was
it's last user.

Signed-off-by: Nir Soffer <address@hidden>
Message-id: address@hidden
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4ee5f4be4f174d23f59a4a1f3fd50f5673f325c5
      
https://github.com/qemu/qemu/commit/4ee5f4be4f174d23f59a4a1f3fd50f5673f325c5
  Author: Thomas Huth <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M tests/qemu-iotests/check

  Log Message:
  -----------
  tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text

When running "make check -j8" or something similar, the iotests are
running in parallel with the other tests. So when they are printing
out "Passed all xx tests" or a similar status message at the end,
it might not be quite clear that this message belongs to the iotests,
since the output might be mixed with the other tests. Thus change the
word "tests" here to "iotests" instead to avoid confusion.

Signed-off-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 3fb643c374233fb16c01d4c27999da505190f7cd
      
https://github.com/qemu/qemu/commit/3fb643c374233fb16c01d4c27999da505190f7cd
  Author: Thomas Huth <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M tests/Makefile.include

  Log Message:
  -----------
  tests/Makefile: Do not print the name of the check-block.sh shell script

The check script is already printing out which iotest is currently
running, so printing out the name of the check-block.sh shell script
looks superfluous here.

Signed-off-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Acked-by: John Snow <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9407cf862ceabddb22ae4bad5a0c8cb5ef6979c5
      
https://github.com/qemu/qemu/commit/9407cf862ceabddb22ae4bad5a0c8cb5ef6979c5
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M tests/qemu-iotests/026.out.nocache

  Log Message:
  -----------
  tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache

qemu-io now prefixes its error and warnings with "qemu-io:".
36b9986b08787019e fixed a lot of iotests output but forget about
026.out.nocache. Fix it too.

Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()")
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 0487861685294660b23bc146e1ebd5304aa8bbe0
      
https://github.com/qemu/qemu/commit/0487861685294660b23bc146e1ebd5304aa8bbe0
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Keep pointer to the CURLState in CURLSocket

A follow-up patch will make curl_multi_do() and curl_multi_read() take a
CURLSocket instead of the CURLState.  They still need the latter,
though, so add a pointer to it to the former.

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: John Snow <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 007f339b1099af46a008dac438ca0943e31dba72
      
https://github.com/qemu/qemu/commit/007f339b1099af46a008dac438ca0943e31dba72
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Keep *socket until the end of curl_sock_cb()

This does not really change anything, but it makes the code a bit easier
to follow once we use @socket as the opaque pointer for
aio_set_fd_handler().

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 948403bcb1c7e71dcbe8ab8479cf3934a0efcbb5
      
https://github.com/qemu/qemu/commit/948403bcb1c7e71dcbe8ab8479cf3934a0efcbb5
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Check completion in curl_multi_do()

While it is more likely that transfers complete after some file
descriptor has data ready to read, we probably should not rely on it.
Better be safe than sorry and call curl_multi_check_completion() in
curl_multi_do(), too, just like it is done in curl_multi_read().

With this change, curl_multi_do() and curl_multi_read() are actually the
same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
handler.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9dbad87d25587ff640ef878f7b6159fc368ff541
      
https://github.com/qemu/qemu/commit/9dbad87d25587ff640ef878f7b6159fc368ff541
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Pass CURLSocket to curl_multi_do()

curl_multi_do_locked() currently marks all sockets as ready.  That is
not only inefficient, but in fact unsafe (the loop is).  A follow-up
patch will change that, but to do so, curl_multi_do_locked() needs to
know exactly which socket is ready; and that is accomplished by this
patch here.

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9abaf9fc474c3dd53e8e119326abc774c977c331
      
https://github.com/qemu/qemu/commit/9abaf9fc474c3dd53e8e119326abc774c977c331
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Report only ready sockets

Instead of reporting all sockets to cURL, only report the one that has
caused curl_multi_do_locked() to be called.  This lets us get rid of the
QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
only safe when the current element is removed in each iteration.  If it
possible for the list to be concurrently modified, we cannot guarantee
that only the current element will be removed.  Therefore, we must not
use QLIST_FOREACH_SAFE() here.

Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: bfb23b480a49114315877aacf700b49453e0f9d9
      
https://github.com/qemu/qemu/commit/bfb23b480a49114315877aacf700b49453e0f9d9
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Handle success in multi_check_completion

Background: As of cURL 7.59.0, it verifies that several functions are
not called from within a callback.  Among these functions is
curl_multi_add_handle().

curl_read_cb() is a callback from cURL and not a coroutine.  Waking up
acb->co will lead to entering it then and there, which means the current
request will settle and the caller (if it runs in the same coroutine)
may then issue the next request.  In such a case, we will enter
curl_setup_preadv() effectively from within curl_read_cb().

Calling curl_multi_add_handle() will then fail and the new request will
not be processed.

Fix this by not letting curl_read_cb() wake up acb->co.  Instead, leave
the whole business of settling the AIOCB objects to
curl_multi_check_completion() (which is called from our timer callback
and our FD handler, so not from any cURL callbacks).

Reported-by: Natalie Gavrielov <address@hidden>
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1740193
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Maxim Levitsky <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: c34dc07f9f01cf686e512f939aece744723072cd
      
https://github.com/qemu/qemu/commit/c34dc07f9f01cf686e512f939aece744723072cd
  Author: Max Reitz <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: Check curl_multi_add_handle()'s return code

If we had done that all along, debugging would have been much simpler.
(Also, I/O errors are better than hangs.)

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Maxim Levitsky <address@hidden>
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: d876bf676f5e7c6aa9ac64555e48cba8734ecb2f
      
https://github.com/qemu/qemu/commit/d876bf676f5e7c6aa9ac64555e48cba8734ecb2f
  Author: Sergio Lopez <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M blockjob.c

  Log Message:
  -----------
  blockjob: update nodes head while removing all bdrv

block_job_remove_all_bdrv() iterates through job->nodes, calling
bdrv_root_unref_child() for each entry. The call to the latter may
reach child_job_[can_]set_aio_ctx(), which will also attempt to
traverse job->nodes, potentially finding entries that where freed
on previous iterations.

To avoid this situation, update job->nodes head on each iteration to
ensure that already freed entries are no longer linked to the list.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
Signed-off-by: Sergio Lopez <address@hidden>
Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Sergio Lopez <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 38e7d54bdc518b5a05a922467304bcace2396945
      
https://github.com/qemu/qemu/commit/38e7d54bdc518b5a05a922467304bcace2396945
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/qcow2-cluster.c

  Log Message:
  -----------
  block/qcow2: Fix corruption introduced by commit 8ac0f15f335

This fixes subtle corruption introduced by luks threaded encryption
in commit 8ac0f15f335

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922

The corruption happens when we do a write that
   * writes to two or more unallocated clusters at once
   * doesn't fully cover the first sector
   * doesn't fully cover the last sector
   * uses luks encryption

In this case, when allocating the new clusters we COW both areas
prior to the write and after the write, and we encrypt them.

The above mentioned commit accidentally made it so we encrypt the
second COW area using the physical cluster offset of the first area.

The problem is that offset_in_cluster in do_perform_cow_encrypt
can be larger that the cluster size, thus cluster_offset
will no longer point to the start of the cluster at which encrypted
area starts.

Next patch in this series will refactor the code to avoid all these
assumptions.

In the bugreport that was triggered by rebasing a luks image to new,
zero filled base, which lot of such writes, and causes some files
with zero areas to contain garbage there instead.
But as described above it can happen elsewhere as well

Signed-off-by: Maxim Levitsky <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 603fbd076c76438b15ec842f0e2d1ba4867dfd00
      
https://github.com/qemu/qemu/commit/603fbd076c76438b15ec842f0e2d1ba4867dfd00
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/qcow2-cluster.c
    M block/qcow2-threads.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  block/qcow2: refactor encryption code

* Change the qcow2_co_{encrypt|decrypt} to just receive full host and
  guest offsets and use this function directly instead of calling
  do_perform_cow_encrypt (which is removed by that patch).

* Adjust qcow2_co_encdec to take full host and guest offsets as well.

* Document the qcow2_co_{encrypt|decrypt} arguments
  to prevent the bug fixed in former commit from hopefully
  happening again.

Signed-off-by: Maxim Levitsky <address@hidden>
Message-id: address@hidden
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
[mreitz: Let perform_cow() return the error value returned by
         qcow2_co_encrypt(), as proposed by Vladimir]
Signed-off-by: Max Reitz <address@hidden>


  Commit: 1825cc0783ccf0ec5d9f0b225a99b340bdd4c68f
      
https://github.com/qemu/qemu/commit/1825cc0783ccf0ec5d9f0b225a99b340bdd4c68f
  Author: Maxim Levitsky <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    A tests/qemu-iotests/263
    A tests/qemu-iotests/263.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  qemu-iotests: Add test for bz #1745922

Signed-off-by: Maxim Levitsky <address@hidden>
Tested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 521db80318d6c749a6f6c5a65a68397af9e3ef16
      
https://github.com/qemu/qemu/commit/521db80318d6c749a6f6c5a65a68397af9e3ef16
  Author: Peter Maydell <address@hidden>
  Date:   2019-09-16 (Mon, 16 Sep 2019)

  Changed paths:
    M block/bochs.c
    M block/cloop.c
    M block/curl.c
    M block/dmg.c
    M block/io.c
    M block/qcow2-cluster.c
    M block/qcow2-threads.c
    M block/qcow2.c
    M block/qcow2.h
    M block/vvfat.c
    M blockjob.c
    M include/block/block.h
    M migration/block.c
    M qemu-img.c
    M tests/Makefile.include
    M tests/qemu-iotests/026.out.nocache
    A tests/qemu-iotests/263
    A tests/qemu-iotests/263.out
    M tests/qemu-iotests/check
    M tests/qemu-iotests/group

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-16' 
into staging

Block patches:
- Fix for block jobs when used with I/O threads
- Fix for a corruption when using qcow2's LUKS encryption mode
- cURL fix
- check-block.sh cleanups (for make check)
- Refactoring

# gpg: Signature made Mon 16 Sep 2019 14:41:15 BST
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Max Reitz <address@hidden>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-09-16:
  qemu-iotests: Add test for bz #1745922
  block/qcow2: refactor encryption code
  block/qcow2: Fix corruption introduced by commit 8ac0f15f335
  blockjob: update nodes head while removing all bdrv
  curl: Check curl_multi_add_handle()'s return code
  curl: Handle success in multi_check_completion
  curl: Report only ready sockets
  curl: Pass CURLSocket to curl_multi_do()
  curl: Check completion in curl_multi_do()
  curl: Keep *socket until the end of curl_sock_cb()
  curl: Keep pointer to the CURLState in CURLSocket
  tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
  tests/Makefile: Do not print the name of the check-block.sh shell script
  tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text
  block: Remove unused masks
  block: Use QEMU_IS_ALIGNED

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/6f214b304457...521db80318d6