[Qemu-commits] [qemu/qemu] 924fe1: aio: fix qemu_bh_schedule() bh->ctx race condition

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 924fe1293c3e7a3c787bbdfb351e7f168caee3e9
      
https://github.com/qemu/qemu/commit/924fe1293c3e7a3c787bbdfb351e7f168caee3e9
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M async.c

  Log Message:
  -----------
  aio: fix qemu_bh_schedule() bh->ctx race condition

qemu_bh_schedule() is supposed to be thread-safe at least the first time
it is called.  Unfortunately this is not quite true:

  bh->scheduled = 1;
  aio_notify(bh->ctx);

Since another thread may run the BH callback once it has been scheduled,
there is a race condition if the callback frees the BH before
aio_notify(bh->ctx) has a chance to run.

Reported-by: Stefan Priebe <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Tested-by: Stefan Priebe <address@hidden>


  Commit: 2572b37a4751cc967582d7d04f21d9bf97187ae5
      
https://github.com/qemu/qemu/commit/2572b37a4751cc967582d7d04f21d9bf97187ae5
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: use BlockDriverState AioContext

Drop the assumption that we're using the main AioContext.  Convert
qemu_aio_wait() to aio_poll() and qemu_bh_new() to aio_bh_new() so the
BlockDriverState AioContext is used.

Note there is still one qemu_aio_wait() left in bdrv_create() but we do
not have a BlockDriverState there and only main loop code invokes this
function.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ed78cda3de92056737364ab3cb748b16f5f17dea
      
https://github.com/qemu/qemu/commit/ed78cda3de92056737364ab3cb748b16f5f17dea
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: acquire AioContext in bdrv_*_all()

bdrv_close_all(), bdrv_commit_all(), bdrv_flush_all(),
bdrv_invalidate_cache_all(), and bdrv_clear_incoming_migration_all() are
called by main loop code and touch all BlockDriverState instances.

Some BlockDriverState instances may be running in another AioContext.
Make sure to acquire the AioContext before closing the BlockDriverState.

This will protect against race conditions once virtio-blk data-plane is
using the BlockDriverState from another AioContext event loop.

Note that this patch does not convert bdrv_drain_all() yet since that
conversion is non-trivial.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9b536adcbefb72090f43c9715ce042e37e47af73
      
https://github.com/qemu/qemu/commit/9b536adcbefb72090f43c9715ce042e37e47af73
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: acquire AioContext in bdrv_drain_all()

Modify bdrv_drain_all() to take into account that BlockDriverState
instances may be running in different AioContexts.

This patch changes the implementation of bdrv_drain_all() while
preserving the semantics.  Previously kicking throttled requests and
checking for pending requests were done across all BlockDriverState
instances in sequence.  Now we process each BlockDriverState in turn,
making sure to acquire and release its AioContext.

This prevents race conditions between the thread executing
bdrv_drain_all() and the thread running the AioContext.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: dcd042282d855edf70df90b7d61d33b515320b7a
      
https://github.com/qemu/qemu/commit/dcd042282d855edf70df90b7d61d33b515320b7a
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: add bdrv_set_aio_context()

Up until now all BlockDriverState instances have used the QEMU main loop
for fd handlers, timers, and BHs.  This is not scalable on SMP guests
and hosts so we need to move to a model with multiple event loops on
different host CPUs.

bdrv_set_aio_context() assigns the AioContext event loop to use for a
particular BlockDriverState.  It first detaches the entire
BlockDriverState graph from the current AioContext and then attaches to
the new AioContext.

This function will be used by virtio-blk data-plane to assign a
BlockDriverState to its IOThread AioContext.  Make
bdrv_aio_set_context() public since data-plane should not include
block_int.h.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7e1efdf0a30faa2b3e560deafe16b8bd12c6f399
      
https://github.com/qemu/qemu/commit/7e1efdf0a30faa2b3e560deafe16b8bd12c6f399
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/blkdebug.c

  Log Message:
  -----------
  blkdebug: use BlockDriverState's AioContext

Drop the assumption that we're using the main AioContext.  Convert
qemu_bh_new() to aio_bh_new() so we use the BlockDriverState's
AioContext.

The .bdrv_detach_aio_context() and .bdrv_attach_aio_context() interfaces
are not needed since no fd handlers, timers, or BHs stay registered when
requests have been drained.

Cc: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 9d94020bc65485fe03a4acabd8559b682b193d07
      
https://github.com/qemu/qemu/commit/9d94020bc65485fe03a4acabd8559b682b193d07
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/blkverify.c

  Log Message:
  -----------
  blkverify: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext.  Convert
qemu_bh_new() to aio_bh_new() and qemu_aio_wait() to aio_poll() so we
use the BlockDriverState's AioContext.

Implement .bdrv_detach/attach_aio_context() interfaces to propagate
detach/attach to BDRVBlkverifyState->test_file.  The block layer takes
care of ->file and ->backing_hd but doesn't know about our ->test_file
BlockDriverState, which is also part of the graph.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 63f0f45f2e89b60ff8245fec81328ddfde42a303
      
https://github.com/qemu/qemu/commit/63f0f45f2e89b60ff8245fec81328ddfde42a303
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/curl.c

  Log Message:
  -----------
  curl: implement .bdrv_detach/attach_aio_context()

The curl block driver uses fd handlers, timers, and BHs.  The fd
handlers and timers are managed on behalf of libcurl, which controls
them using callback functions that the block driver implements.

The simplest way to implement .bdrv_detach/attach_aio_context() is to
clean up libcurl in the old event loop and initialize it again in the
new event loop.  We do not need to keep track of anything since there
are no pending requests when the AioContext is changed.

Also make sure to use aio_set_fd_handler() instead of
qemu_aio_set_fd_handler() and aio_bh_new() instead of qemu_bh_new() so
the current AioContext is passed in.

Cc: Alexander Graf <address@hidden>
Cc: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>


  Commit: 6ee50af24b1478452fbbfc1cd8d78227c0e91cde
      
https://github.com/qemu/qemu/commit/6ee50af24b1478452fbbfc1cd8d78227c0e91cde
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  gluster: use BlockDriverState's AioContext

Drop the assumption that we're using the main AioContext.  Use
aio_bh_new() instead of qemu_bh_new().

The .bdrv_detach_aio_context() and .bdrv_attach_aio_context() interfaces
are not needed since no fd handlers, timers, or BHs stay registered when
requests have been drained.

Cc: Bharata B Rao <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 80cf6257668ec78537b02af44e059bdc46c30416
      
https://github.com/qemu/qemu/commit/80cf6257668ec78537b02af44e059bdc46c30416
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext for Linux
AIO.  Convert qemu_aio_set_fd_handler() to aio_set_fd_handler() and
timer_new_ms() to aio_timer_new().

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the fd and timer from the old to the new AioContext.

Cc: Peter Lieven <address@hidden>
Cc: Ronnie Sahlberg <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Peter Lieven <address@hidden>


  Commit: 69447cd8f361dbb6c752bd18b15322d27493ca91
      
https://github.com/qemu/qemu/commit/69447cd8f361dbb6c752bd18b15322d27493ca91
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/nbd-client.c
    M block/nbd-client.h
    M block/nbd.c

  Log Message:
  -----------
  nbd: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext.  Convert
qemu_aio_set_fd_handler() calls to aio_set_fd_handler().

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the socket fd handler from the old to the new
AioContext.

Acked-by: Paolo Bonzini <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 471799d135a45b5a09f43bc103b5eb710548bb87
      
https://github.com/qemu/qemu/commit/471799d135a45b5a09f43bc103b5eb710548bb87
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/nfs.c

  Log Message:
  -----------
  nfs: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext.  The following
functions need to be converted:
 * qemu_bh_new() -> aio_bh_new()
 * qemu_aio_set_fd_handler() -> aio_set_fd_handler()
 * qemu_aio_wait() -> aio_poll()

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the fd handler from the old to the new AioContext.

Cc: Peter Lieven <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Peter Lieven <address@hidden>


  Commit: a8c868c386d4cbdd0fc0a9423c6683880f6e75e4
      
https://github.com/qemu/qemu/commit/a8c868c386d4cbdd0fc0a9423c6683880f6e75e4
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/qed-table.c
    M block/qed.c

  Log Message:
  -----------
  qed: use BlockDriverState's AioContext

Drop the assumption that we're using the main AioContext.  Convert
qemu_bh_new() to aio_bh_new() and qemu_aio_wait() to aio_poll() so we're
using the BlockDriverState's AioContext.

Implement .bdrv_detach/attach_aio_context() interfaces to move the
QED_F_NEED_CHECK timer from the old AioContext to the new one.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e3625d3d6ae8029fc20262ac3848421ec281446f
      
https://github.com/qemu/qemu/commit/e3625d3d6ae8029fc20262ac3848421ec281446f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/quorum.c

  Log Message:
  -----------
  quorum: implement .bdrv_detach/attach_aio_context()

Implement .bdrv_detach/attach_aio_context() interfaces to propagate
detach/attach to BDRVQuorumState->bs[] children.  The block layer takes
care of ->file and ->backing_hd but doesn't know about our ->bs[]
BlockDriverStates, which is also part of the graph.

Reviewed-by: Benoît Canet <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: c2f3426c9bba0195b15a919587f794b110a2dcfc
      
https://github.com/qemu/qemu/commit/c2f3426c9bba0195b15a919587f794b110a2dcfc
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/linux-aio.c
    M block/raw-aio.h
    M block/raw-posix.c

  Log Message:
  -----------
  block/raw-posix: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext for Linux AIO.
Convert the Linux AIO event notifier to use aio_set_event_notifier().

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the event notifier handler from the old to the new
AioContext.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: abd269b7cf1f084a067731acb8f3272c193cb5f0
      
https://github.com/qemu/qemu/commit/abd269b7cf1f084a067731acb8f3272c193cb5f0
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/linux-aio.c
    M block/raw-aio.h
    M block/raw-posix.c

  Log Message:
  -----------
  block/linux-aio: fix memory and fd leak

Hot unplugging -drive aio=native,file=test.img,format=raw images leaves
the Linux AIO event notifier and struct qemu_laio_state allocated.
Luckily nothing will use the event notifier after the BlockDriverState
has been closed so the handler function is never called.

It's still worth fixing this resource leak.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 99cc598924d9686e497fa9ddd7889d6be27f3084
      
https://github.com/qemu/qemu/commit/99cc598924d9686e497fa9ddd7889d6be27f3084
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/raw-aio.h
    M block/raw-win32.c
    M block/win32-aio.c

  Log Message:
  -----------
  block/raw-win32: create one QEMUWin32AIOState per BDRVRawState

Each QEMUWin32AIOState event notifier is associated with an AioContext.
Since BlockDriverState instances can use different AioContexts we cannot
continue to use a global QEMUWin32AIOState.

Let each BDRVRawState have its own QEMUWin32AIOState and free it when
BDRVRawState is closed.

Reviewed-by: Paolo Bonzini <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 85ebd381fdae2d551cd6a0ffff47ab08734edf12
      
https://github.com/qemu/qemu/commit/85ebd381fdae2d551cd6a0ffff47ab08734edf12
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/raw-aio.h
    M block/raw-win32.c
    M block/win32-aio.c

  Log Message:
  -----------
  block/raw-win32: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext for raw-win32.
Convert the aio-win32 code to support detach/attach and replace
qemu_aio_wait() with aio_poll().

The .bdrv_detach/attach_aio_context() interfaces move the aio-win32
event notifier from the old to the new AioContext.

Cc: Paolo Bonzini <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ea8001914924331549e9b876bcaeac2dc80db2b6
      
https://github.com/qemu/qemu/commit/ea8001914924331549e9b876bcaeac2dc80db2b6
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/rbd.c

  Log Message:
  -----------
  rbd: use BlockDriverState's AioContext

Drop the assumption that we're using the main AioContext.  Convert
qemu_bh_new() to aio_bh_new() and qemu_aio_wait() to aio_poll().

The .bdrv_detach_aio_context() and .bdrv_attach_aio_context() interfaces
are not needed since no fd handlers, timers, or BHs stay registered when
requests have been drained.

Cc: Josh Durgin <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Josh Durgin <address@hidden>


  Commit: 84390bed59f5a8ead499c4501f54c9b8a747c9d4
      
https://github.com/qemu/qemu/commit/84390bed59f5a8ead499c4501f54c9b8a747c9d4
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: implement .bdrv_detach/attach_aio_context()

Drop the assumption that we're using the main AioContext.  Convert
qemu_aio_set_fd_handler() to aio_set_fd_handler() and qemu_aio_wait() to
aio_poll().

The .bdrv_detach/attach_aio_context() interfaces also need to be
implemented to move the socket fd handler from the old to the new
AioContext.

Cc: MORITA Kazutaka <address@hidden>
Acked-by: Liu Yuan <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 2af0b2005669ee3572a19499c51a499114a8ec50
      
https://github.com/qemu/qemu/commit/2af0b2005669ee3572a19499c51a499114a8ec50
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/ssh.c

  Log Message:
  -----------
  ssh: use BlockDriverState's AioContext

Drop the assumption that we're using the main AioContext.  Use
bdrv_get_aio_context() to register fd handlers in the right AioContext
for this BlockDriverState.

The .bdrv_detach_aio_context() and .bdrv_attach_aio_context() interfaces
are not needed since no fd handlers, timers, or BHs stay registered when
requests have been drained.

For now this doesn't make much difference but will allow ssh to work in
IOThread instances in the future.

Acked-by: Richard W.M. Jones <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: c75f3bdf46f74c9f1c199ace3d2b291e430dcda3
      
https://github.com/qemu/qemu/commit/c75f3bdf46f74c9f1c199ace3d2b291e430dcda3
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/vmdk.c

  Log Message:
  -----------
  vmdk: implement .bdrv_detach/attach_aio_context()

Implement .bdrv_detach/attach_aio_context() interfaces to propagate
detach/attach to BDRVVmdkState->extents[].file.  The block layer takes
care of ->file and ->backing_hd but doesn't know about our extents
BlockDriverStates, which is also part of the graph.

Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 580b6b2aa2fea42ee42921ef5812b40224ab1b49
      
https://github.com/qemu/qemu/commit/580b6b2aa2fea42ee42921ef5812b40224ab1b49
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/dataplane/virtio-blk.c

  Log Message:
  -----------
  dataplane: use the QEMU block layer for I/O

Stop using a custom Linux AIO request queue from ioq.h and instead use
the QEMU block layer for I/O.

This patch adjusts the VirtIOBlockRequest struct with fields needed for
bdrv_aio_readv()/bdrv_aio_writev().  ioq.h used struct iovec and struct
iocb, which we don't need directly anymore.

Modify dataplane start/stop to set the AioContext on the
BlockDriverState.  We also no longer need to get the raw-posix file
descriptor.  This means image formats are now supported with dataplane!

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 0d6ecec2526d26ada9f4e44d27686a40ab9c087b
      
https://github.com/qemu/qemu/commit/0d6ecec2526d26ada9f4e44d27686a40ab9c087b
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/dataplane/Makefile.objs
    R hw/block/dataplane/ioq.c
    R hw/block/dataplane/ioq.h

  Log Message:
  -----------
  dataplane: delete IOQueue since it is no longer used

This custom Linux AIO request queue is no longer used by virtio-blk
data-plane.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8d242f0ed5908333035dd5a8a94c06d7fe3b765c
      
https://github.com/qemu/qemu/commit/8d242f0ed5908333035dd5a8a94c06d7fe3b765c
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/dataplane/virtio-blk.c

  Log Message:
  -----------
  dataplane: implement async flush

Stop using the raw-posix file descriptor for synchronous
qemu_fdatasync().  Use bdrv_aio_flush() instead and drop the
VirtIOBlockDataPlane->fd field.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 76ef2cf5493a215efc351f48ae7094d6c183fcac
      
https://github.com/qemu/qemu/commit/76ef2cf5493a215efc351f48ae7094d6c183fcac
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block/raw-posix.c
    M include/block/block.h

  Log Message:
  -----------
  raw-posix: drop raw_get_aio_fd() since it is no longer used

virtio-blk data-plane now uses the QEMU block layer for I/O.  We do not
need raw_get_aio_fd() anymore.  It was a layering violation anyway, so
let's get rid of it.

Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: db519cba8713fb49aa5233e8debe61dccdd3a57f
      
https://github.com/qemu/qemu/commit/db519cba8713fb49aa5233e8debe61dccdd3a57f
  Author: Fam Zheng <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Move declaration of bdrv_get_aio_context to block.h

block_int.h is for block layer and block drivers, other code shouldn't
include it. But similar to bdrv_set_aio_context, bdrv_get_aio_context
should also be accessible from outside of block layer.

Move it.

Signed-off-by: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 6d7e73d62fa32813b6f6a3575db2e9b5e0d43387
      
https://github.com/qemu/qemu/commit/6d7e73d62fa32813b6f6a3575db2e9b5e0d43387
  Author: Fam Zheng <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/dataplane/virtio-blk.c
    M hw/block/virtio-blk.c

  Log Message:
  -----------
  virtio-blk: Allow config-wce in dataplane

Dataplane now uses block layer. Protect bdrv_set_enable_write_cache with
aio_context_acquire and aio_context_release, so we can enable config-wce
to allow guest to modify the write cache online.

Signed-off-by: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5a05cbeeaaa2ec463d48c0026e8e6be243ea0bab
      
https://github.com/qemu/qemu/commit/5a05cbeeaaa2ec463d48c0026e8e6be243ea0bab
  Author: Fam Zheng <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/virtio-blk.c
    M include/hw/virtio/virtio-blk.h

  Log Message:
  -----------
  virtio-blk: Factor out virtio_blk_handle_scsi_req from virtio_blk_handle_scsi

The common logic to process a scsi request in a VirtQueueElement is
extracted to a function to share with dataplane.

This makes VirtIOBlockReq.scsi unused, so drop it.

Signed-off-by: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: c9f87b20b996b1005b646281195d61412ba2b844
      
https://github.com/qemu/qemu/commit/c9f87b20b996b1005b646281195d61412ba2b844
  Author: Fam Zheng <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M hw/block/dataplane/virtio-blk.c

  Log Message:
  -----------
  dataplane: Support VIRTIO_BLK_T_SCSI_CMD

Signed-off-by: Fam Zheng <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 13af91ebf08d463d3b025cd396d4d11caceac02d
      
https://github.com/qemu/qemu/commit/13af91ebf08d463d3b025cd396d4d11caceac02d
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block.c
    M include/qemu/throttle.h
    M tests/test-throttle.c
    M util/throttle.c

  Log Message:
  -----------
  throttle: add throttle_detach/attach_aio_context()

Block I/O throttling uses timers and currently always adds them to the
main loop.  Throttling will break if bdrv_set_aio_context() is used to
move a BlockDriverState to a different AioContext.

This patch adds throttle_detach/attach_aio_context() interfaces so the
throttling timers and uses them to move timers to the new AioContext.
Note that bdrv_set_aio_context() already drains all requests so we're
sure no throttled requests are pending.

The test cases need to be updated since the throttle_init() interface
has changed.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>


  Commit: 22524f726210f38298dfb1cefa30257133ad6b8c
      
https://github.com/qemu/qemu/commit/22524f726210f38298dfb1cefa30257133ad6b8c
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M tests/test-throttle.c

  Log Message:
  -----------
  throttle: add detach/attach test case

Add a test case that checks the timer is really removed/added by the
detach/attach functions.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>


  Commit: b15446fdbf4ac2b29f6ee5080630a80715abfc20
      
https://github.com/qemu/qemu/commit/b15446fdbf4ac2b29f6ee5080630a80715abfc20
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M blockdev.c

  Log Message:
  -----------
  blockdev: acquire AioContext in block_set_io_throttle

The block_set_io_throttle QMP and HMP commands modify I/O throttling
limits for block devices.

Acquire the BlockDriverState's AioContext to protect against race
conditions with an IOThread that is running I/O for this device.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>


  Commit: 1ac362cdbd799eb8165e2e3bb5cd5aa38b1baae3
      
https://github.com/qemu/qemu/commit/1ac362cdbd799eb8165e2e3bb5cd5aa38b1baae3
  Author: chai wen <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M block-migration.c

  Log Message:
  -----------
  block: fix wrong order in live block migration setup

The function init_blk_migration is better to be called before
set_dirty_tracking as the reasons below.

If we want to track dirty blocks via dirty_maps on a BlockDriverState
when doing live block-migration, its correspoding 'BlkMigDevState' should be
added to block_mig_state.bmds_list first for subsequent processing.
Otherwise set_dirty_tracking will do nothing on an empty list than allocating
dirty_bitmaps for them. And bdrv_get_dirty_count will access the
bmds->dirty_maps directly, then there would be a segfault triggered.

If the set_dirty_tracking fails, qemu_savevm_state_cancel will handle
the cleanup of init_blk_migration automatically.

Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: chai wen <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d6635c4dbbcfca7f5bd379dd970617fc3430428f
      
https://github.com/qemu/qemu/commit/d6635c4dbbcfca7f5bd379dd970617fc3430428f
  Author: Max Reitz <address@hidden>
  Date:   2014-06-04 (Wed, 04 Jun 2014)

  Changed paths:
    M qemu-img.c
    M qemu-img.texi

  Log Message:
  -----------
  qemu-img: Document check exit codes

The exit code 63 (check not supported by image format) was not even
documented in the comment above the check command in the source code;
add it, as it does indeed seem useful.

Also, document all of check's exit codes in the manpage.

Signed-off-by: Max Reitz <address@hidden>
Reported-by: Markus Armbruster <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 405a27640b33c31ccef4001b3f3936b8c9d2218f
      
https://github.com/qemu/qemu/commit/405a27640b33c31ccef4001b3f3936b8c9d2218f
  Author: Kevin Wolf <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M block/rbd.c

  Log Message:
  -----------
  rbd: Fix leaks in rbd_start_aio() error path

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Benoit Canet <address@hidden>
Reviewed-by: Josh Durgin <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: b544c1aba8681c2fe5d6715fbd37cf6caf1bc7bb
      
https://github.com/qemu/qemu/commit/b544c1aba8681c2fe5d6715fbd37cf6caf1bc7bb
  Author: Hitoshi Mitake <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: fix vdi object update after live snapshot

sheepdog driver should decide a write request is COW or not based on inode
object which is active when the write request is issued.

Example of wrong inode update path in the previous driver:
1. drier issues an ordinal write request to an existing object
2. user creates a snapshot of the VDI before the write request is completed
3. the respones for the request is RDONLY, because the VDI is already a snapshot
4. the driver reload an inode object of the new active VDI, then issues a write
   request again
5. the second write request can be completed
6. driver decide the request is COW or not with the below conditional branch:
          if (s->inode.data_vdi_id[idx] != s->inode.vdi_id) {
7. the ID of the written object and VID of the new active VDI is different, so
   the driver updates data_vdi_id[idx] and writes inode object
8. the existing object cannot be seen by the new active VDI, it results object
   leaking

Cc: Kevin Wolf <address@hidden>
Cc: Stefan Hajnoczi <address@hidden>
Cc: Liu Yuan <address@hidden>
Cc: MORITA Kazutaka <address@hidden>
Signed-off-by: Hitoshi Mitake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5d039baba411ef90b53150ae394575b25c9bfb74
      
https://github.com/qemu/qemu/commit/5d039baba411ef90b53150ae394575b25c9bfb74
  Author: Hitoshi Mitake <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: reload only header in a case of live snapshot

sheepdog driver doesn't need to read data_vdi_id[] when a live snapshot is
created.

Cc: Kevin Wolf <address@hidden>
Cc: Stefan Hajnoczi <address@hidden>
Cc: Liu Yuan <address@hidden>
Cc: MORITA Kazutaka <address@hidden>
Signed-off-by: Hitoshi Mitake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d34bda716a7da18ee6f239ce81240ce063ac844e
      
https://github.com/qemu/qemu/commit/d34bda716a7da18ee6f239ce81240ce063ac844e
  Author: Benoît Canet <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M qapi-schema.json
    A qapi/common.json

  Log Message:
  -----------
  qapi: Extract qapi/common.json definitions

Signed-off-by: Benoit Canet <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5db15096d047f0ec92bd5f1680343ea7acbed4e5
      
https://github.com/qemu/qemu/commit/5db15096d047f0ec92bd5f1680343ea7acbed4e5
  Author: Benoît Canet <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M qapi-schema.json
    A qapi/block-core.json
    A qapi/block.json

  Log Message:
  -----------
  qapi: create two block related json modules

qapi/block-core.json contains block definitions unrelated to emulation.

qapi/block.json is a superset of the previous and contains definitions related
to emulation.

The purpose of these extractions is to be able to hook qapi/block-core.json
generated code on qemu-nbd.

Signed-off-by: Benoit Canet <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 1ad166b612c2aca517430d88f4a3d60f96bb6756
      
https://github.com/qemu/qemu/commit/1ad166b612c2aca517430d88f4a3d60f96bb6756
  Author: Benoît Canet <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M qapi-schema.json
    M qapi/block-core.json

  Log Message:
  -----------
  qapi: Extract qapi/block-core.json definitions

Signed-off-by: Benoit Canet <address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 2e95fa17196aa12e7e522925ad420000162153d0
      
https://github.com/qemu/qemu/commit/2e95fa17196aa12e7e522925ad420000162153d0
  Author: Benoît Canet <address@hidden>
  Date:   2014-06-06 (Fri, 06 Jun 2014)

  Changed paths:
    M qapi-schema.json
    M qapi/block.json

  Log Message:
  -----------
  qapi: Extract qapi/block.json definitions

Signed-off-by: Benoit Canet <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5dfc05cb1d342f081df7009703559b29dadc33e9
      
https://github.com/qemu/qemu/commit/5dfc05cb1d342f081df7009703559b29dadc33e9
  Author: Peter Maydell <address@hidden>
  Date:   2014-06-09 (Mon, 09 Jun 2014)

  Changed paths:
    M async.c
    M block-migration.c
    M block.c
    M block/blkdebug.c
    M block/blkverify.c
    M block/curl.c
    M block/gluster.c
    M block/iscsi.c
    M block/linux-aio.c
    M block/nbd-client.c
    M block/nbd-client.h
    M block/nbd.c
    M block/nfs.c
    M block/qed-table.c
    M block/qed.c
    M block/quorum.c
    M block/raw-aio.h
    M block/raw-posix.c
    M block/raw-win32.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/vmdk.c
    M block/win32-aio.c
    M blockdev.c
    M hw/block/dataplane/Makefile.objs
    R hw/block/dataplane/ioq.c
    R hw/block/dataplane/ioq.h
    M hw/block/dataplane/virtio-blk.c
    M hw/block/virtio-blk.c
    M include/block/block.h
    M include/block/block_int.h
    M include/hw/virtio/virtio-blk.h
    M include/qemu/throttle.h
    M qapi-schema.json
    A qapi/block-core.json
    A qapi/block.json
    A qapi/common.json
    M qemu-img.c
    M qemu-img.texi
    M tests/test-throttle.c
    M util/throttle.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into 
staging

Block pull request

# gpg: Signature made Fri 06 Jun 2014 17:08:50 BST using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>"
# gpg:                 aka "Stefan Hajnoczi <address@hidden>"

* remotes/stefanha/tags/block-pull-request: (42 commits)
  qapi: Extract qapi/block.json definitions
  qapi: Extract qapi/block-core.json definitions
  qapi: create two block related json modules
  qapi: Extract qapi/common.json definitions
  sheepdog: reload only header in a case of live snapshot
  sheepdog: fix vdi object update after live snapshot
  rbd: Fix leaks in rbd_start_aio() error path
  qemu-img: Document check exit codes
  block: fix wrong order in live block migration setup
  blockdev: acquire AioContext in block_set_io_throttle
  throttle: add detach/attach test case
  throttle: add throttle_detach/attach_aio_context()
  dataplane: Support VIRTIO_BLK_T_SCSI_CMD
  virtio-blk: Factor out virtio_blk_handle_scsi_req from virtio_blk_handle_scsi
  virtio-blk: Allow config-wce in dataplane
  block: Move declaration of bdrv_get_aio_context to block.h
  raw-posix: drop raw_get_aio_fd() since it is no longer used
  dataplane: implement async flush
  dataplane: delete IOQueue since it is no longer used
  dataplane: use the QEMU block layer for I/O
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/959e41473f21...5dfc05cb1d34