[Qemu-commits] [qemu/qemu] c9b7d9: virtio: increase virtqueue size for virtio-scsi an...

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: c9b7d9ec21dfca716f0bb3b68dee75660d86629c
      
https://github.com/qemu/qemu/commit/c9b7d9ec21dfca716f0bb3b68dee75660d86629c
  Author: Denis Plotnikov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M hw/block/virtio-blk.c
    M hw/core/machine.c
    M hw/scsi/virtio-scsi.c

  Log Message:
  -----------
  virtio: increase virtqueue size for virtio-scsi and virtio-blk

The goal is to reduce the amount of requests issued by a guest on
1M reads/writes. This rises the performance up to 4% on that kind of
disk access pattern.

The maximum chunk size to be used for the guest disk accessing is
limited with seg_max parameter, which represents the max amount of
pices in the scatter-geather list in one guest disk request.

Since seg_max is virqueue_size dependent, increasing the virtqueue
size increases seg_max, which, in turn, increases the maximum size
of data to be read/write from a guest disk.

More details in the original problem statment:
https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg03721.html

Suggested-by: Denis V. Lunev <address@hidden>
Signed-off-by: Denis Plotnikov <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: f25c0b547916962d0b1be260b5b643287bea0851
      
https://github.com/qemu/qemu/commit/f25c0b547916962d0b1be260b5b643287bea0851
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M util/aio-posix.c

  Log Message:
  -----------
  aio-posix: avoid reacquiring rcu_read_lock() when polling

The first rcu_read_lock/unlock() is expensive.  Nested calls are cheap.

This optimization increases IOPS from 73k to 162k with a Linux guest
that has 2 virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32
devices.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8c3570e33954d26675ec6fd224ede02763dfbd1d
      
https://github.com/qemu/qemu/commit/8c3570e33954d26675ec6fd224ede02763dfbd1d
  Author: Paolo Bonzini <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/qemu/queue.h
    M include/qemu/rcu_queue.h
    M tests/Makefile.include
    M tests/test-rcu-list.c
    A tests/test-rcu-slist.c

  Log Message:
  -----------
  rcu_queue: add QSLIST functions

QSLIST is the only family of lists for which we do not have RCU-friendly 
accessors,
add them.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 8c6b0356b53977bcfdea5299db07884915425b0c
      
https://github.com/qemu/qemu/commit/8c6b0356b53977bcfdea5299db07884915425b0c
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/block/aio.h
    M tests/test-aio.c
    M util/async.c

  Log Message:
  -----------
  util/async: make bh_aio_poll() O(1)

The ctx->first_bh list contains all created BHs, including those that
are not scheduled.  The list is iterated by the event loop and therefore
has O(n) time complexity with respected to the number of created BHs.

Rewrite BHs so that only scheduled or deleted BHs are enqueued.
Only BHs that actually require action will be iterated.

One semantic change is required: qemu_bh_delete() enqueues the BH and
therefore invokes aio_notify().  The
tests/test-aio.c:test_source_bh_delete_from_cb() test case assumed that
g_main_context_iteration(NULL, false) returns false after
qemu_bh_delete() but it now returns true for one iteration.  Fix up the
test case.

This patch makes aio_compute_timeout() and aio_bh_poll() drop from a CPU
profile reported by perf-top(1).  Previously they combined to 9% CPU
utilization when AioContext polling is commented out and the guest has 2
virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ff29ed3a331d0cd26bcd30f7cd6c0c96c7d44eed
      
https://github.com/qemu/qemu/commit/ff29ed3a331d0cd26bcd30f7cd6c0c96c7d44eed
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M util/aio-posix.c

  Log Message:
  -----------
  aio-posix: fix use after leaving scope in aio_poll()

epoll_handler is a stack variable and must not be accessed after it goes
out of scope:

      if (aio_epoll_check_poll(ctx, pollfds, npfd, timeout)) {
          AioHandler epoll_handler;
          ...
          add_pollfd(&epoll_handler);
          ret = aio_epoll(ctx, pollfds, npfd, timeout);
      } ...

  ...

  /* if we have any readable fds, dispatch event */
  if (ret > 0) {
      for (i = 0; i < npfd; i++) {
          nodes[i]->pfd.revents = pollfds[i].revents;
      }
  }

nodes[0] is &epoll_handler, which has already gone out of scope.

There is no need to use pollfds[] for epoll.  We don't need an
AioHandler for the epoll fd.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Sergio Lopez <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ca8c6b22754b0f17818b1d1910d31f0aa1a49cc7
      
https://github.com/qemu/qemu/commit/ca8c6b22754b0f17818b1d1910d31f0aa1a49cc7
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M util/aio-posix.c

  Log Message:
  -----------
  aio-posix: don't pass ns timeout to epoll_wait()

Don't pass the nanosecond timeout into epoll_wait(), which expects
milliseconds.

The epoll_wait() timeout value does not matter if qemu_poll_ns()
determined that the poll fd is ready, but passing a value in the wrong
units is still ugly.  Pass a 0 timeout to epoll_wait() instead.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Sergio Lopez <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 195ed8cb365edeb0d0a70a2ffdeb7a073f9a8117
      
https://github.com/qemu/qemu/commit/195ed8cb365edeb0d0a70a2ffdeb7a073f9a8117
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M block.c
    M chardev/spice.c
    M include/qemu/queue.h

  Log Message:
  -----------
  qemu/queue.h: add QLIST_SAFE_REMOVE()

QLIST_REMOVE() assumes the element is in a list.  It also leaves the
element's linked list pointers dangling.

Introduce a safe version of QLIST_REMOVE() and convert open-coded
instances of this pattern.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Sergio Lopez <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 4749079ce033a94784cbe20a661abeac598ff057
      
https://github.com/qemu/qemu/commit/4749079ce033a94784cbe20a661abeac598ff057
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/block/aio.h
    M include/qemu/queue.h
    M util/aio-posix.c

  Log Message:
  -----------
  aio-posix: make AioHandler deletion O(1)

It is not necessary to scan all AioHandlers for deletion.  Keep a list
of deleted handlers instead of scanning the full list of all handlers.

The AioHandler->deleted field can be dropped.  Let's check if the
handler has been inserted into the deleted list instead.  Add a new
QLIST_IS_INSERTED() API for this check.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Sergio Lopez <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7391d34c3cca09c0bb0140275839c6619b86ec0f
      
https://github.com/qemu/qemu/commit/7391d34c3cca09c0bb0140275839c6619b86ec0f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M util/aio-posix.c

  Log Message:
  -----------
  aio-posix: make AioHandler dispatch O(1) with epoll

File descriptor monitoring is O(1) with epoll(7), but
aio_dispatch_handlers() still scans all AioHandlers instead of
dispatching just those that are ready.  This makes aio_poll() O(n) with
respect to the total number of registered handlers.

Add a local ready_list to aio_poll() so that each nested aio_poll()
builds a list of handlers ready to be dispatched.  Since file descriptor
polling is level-triggered, nested aio_poll() calls also see fds that
were ready in the parent but not yet dispatched.  This guarantees that
nested aio_poll() invocations will dispatch all fds, even those that
became ready before the nested invocation.

Since only handlers ready to be dispatched are placed onto the
ready_list, the new aio_dispatch_ready_handlers() function provides O(1)
dispatch.

Note that AioContext polling is still O(n) and currently cannot be fully
disabled.  This still needs to be fixed before aio_poll() is fully O(1).

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Sergio Lopez <address@hidden>
Message-id: address@hidden
[Fix compilation error on macOS where there is no epoll(87).  The
aio_epoll() prototype was out of date and aio_add_ready_list() needed to
be moved outside the ifdef.
--Stefan]
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: bac068e0648c1f5c37f6a0a9423b8aa55e8c09c2
      
https://github.com/qemu/qemu/commit/bac068e0648c1f5c37f6a0a9423b8aa55e8c09c2
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile.objs
    M Makefile.target
    M scripts/checkpatch.pl
    M scripts/get_maintainer.pl
    A softmmu/Makefile.objs
    A softmmu/vl.c
    R vl.c

  Log Message:
  -----------
  softmmu: move vl.c to softmmu/

Move vl.c to a separate directory, similar to linux-user/
Update the chechpatch and get_maintainer scripts, since they relied on
/vl.c for top_of_tree checks.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 7b73386222626608f843ca4773426dce4ebcc73a
      
https://github.com/qemu/qemu/commit/7b73386222626608f843ca4773426dce4ebcc73a
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile.target
    M include/sysemu/sysemu.h
    M softmmu/Makefile.objs
    A softmmu/main.c
    M softmmu/vl.c

  Log Message:
  -----------
  softmmu: split off vl.c:main() into main.c

A program might rely on functions implemented in vl.c, but implement its
own main(). By placing main into a separate source file, there are no
complaints about duplicate main()s when linking against vl.o. For
example, the virtual-device fuzzer uses a main() provided by libfuzzer,
and needs to perform some initialization before running the softmmu
initialization. Now, main simply calls three vl.c functions which
handle the guest initialization, main loop and cleanup.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 46a07579ebb081493618bfa00ef8e241cd0dcc4f
      
https://github.com/qemu/qemu/commit/46a07579ebb081493618bfa00ef8e241cd0dcc4f
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M util/module.c

  Log Message:
  -----------
  module: check module wasn't already initialized

The virtual-device fuzzer must initialize QOM, prior to running
vl:qemu_init, so that it can use the qos_graph to identify the arguments
required to initialize a guest for libqos-assisted fuzzing. This change
prevents errors when vl:qemu_init tries to (re)initialize the previously
initialized QOM module.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e785e50a5eb37e143bbe68b1693753b9bcfba005
      
https://github.com/qemu/qemu/commit/e785e50a5eb37e143bbe68b1693753b9bcfba005
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/qemu/module.h

  Log Message:
  -----------
  fuzz: add FUZZ_TARGET module type

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e731d083e34b4dbebf0870c137df4405e4ae8319
      
https://github.com/qemu/qemu/commit/e731d083e34b4dbebf0870c137df4405e4ae8319
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/sysemu/qtest.h
    M qtest.c

  Log Message:
  -----------
  qtest: add qtest_server_send abstraction

qtest_server_send is a function pointer specifying the handler used to
transmit data to the qtest client. In the standard configuration, this
calls the CharBackend handler, but now it is possible for other types of
handlers, e.g direct-function calls if the qtest client and server
exist within the same process (inproc)

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Acked-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 075334810b3c795c7120eecaf18945befbb816c6
      
https://github.com/qemu/qemu/commit/075334810b3c795c7120eecaf18945befbb816c6
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/libqtest.c

  Log Message:
  -----------
  libqtest: add a layer of abstraction to send/recv

This makes it simple to swap the transport functions for qtest commands
to and from the qtest client. For example, now it is possible to
directly pass qtest commands to a server handler that exists within the
same process, without the standard way of writing to a file descriptor.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: ca5d464151c72695a960d0f493f2fe7c083e468f
      
https://github.com/qemu/qemu/commit/ca5d464151c72695a960d0f493f2fe7c083e468f
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/libqtest.c
    M tests/qtest/libqtest.h

  Log Message:
  -----------
  libqtest: make bufwrite rely on the TransportOps

When using qtest "in-process" communication, qtest_sendf directly calls
a function in the server (qtest.c). Previously, bufwrite used
socket_send, which bypasses the TransportOps enabling the call into
qtest.c. This change replaces the socket_send calls with ops->send,
maintaining the benefits of the direct socket_send call, while adding
support for in-process qtest calls.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 0bd9aef89ba941b41773d9dbfa94433c2b7d00de
      
https://github.com/qemu/qemu/commit/0bd9aef89ba941b41773d9dbfa94433c2b7d00de
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M include/sysemu/qtest.h
    M qtest.c

  Log Message:
  -----------
  qtest: add in-process incoming command handler

The handler allows a qtest client to send commands to the server by
directly calling a function, rather than using a file/CharBackend

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 39397a9a76eb02ad8a772f43446fdb3344093c35
      
https://github.com/qemu/qemu/commit/39397a9a76eb02ad8a772f43446fdb3344093c35
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/libqos/i2c.c
    M tests/qtest/libqos/i2c.h
    M tests/qtest/pca9552-test.c

  Log Message:
  -----------
  libqos: rename i2c_send and i2c_recv

The names i2c_send and i2c_recv collide with functions defined in
hw/i2c/core.c. This causes an error when linking against libqos and
softmmu simultaneously (for example when using qtest inproc). Rename the
libqos functions to avoid this.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Acked-by: Thomas Huth <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 92ecf9be906edfbde10f651b9165e51c600924fc
      
https://github.com/qemu/qemu/commit/92ecf9be906edfbde10f651b9165e51c600924fc
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/Makefile.include

  Log Message:
  -----------
  libqos: split qos-test and libqos makefile vars

Most qos-related objects were specified in the qos-test-obj-y variable.
qos-test-obj-y also included qos-test.o which defines a main().
This made it difficult to repurpose qos-test-obj-y to link anything
beside tests/qos-test against libqos. This change separates objects that
are libqos-specific and ones that are qos-test specific into different
variables.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: f62a0bff6a5266e7d434de2e1b01fb1f925a9796
      
https://github.com/qemu/qemu/commit/f62a0bff6a5266e7d434de2e1b01fb1f925a9796
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/Makefile.include
    A tests/qtest/libqos/qos_external.c
    A tests/qtest/libqos/qos_external.h
    M tests/qtest/qos-test.c

  Log Message:
  -----------
  libqos: move useful qos-test funcs to qos_external

The moved functions are not specific to qos-test and might be useful
elsewhere. For example the virtual-device fuzzer makes use of them for
qos-assisted fuzz-targets.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 5f6fd09a9729d31225b6eaec5df05d19a5bdfda4
      
https://github.com/qemu/qemu/commit/5f6fd09a9729d31225b6eaec5df05d19a5bdfda4
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M MAINTAINERS
    A tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/fuzz.c
    A tests/qtest/fuzz/fuzz.h

  Log Message:
  -----------
  fuzz: add fuzzer skeleton

tests/fuzz/fuzz.c serves as the entry point for the virtual-device
fuzzer. Namely, libfuzzer invokes the LLVMFuzzerInitialize and
LLVMFuzzerTestOneInput functions, both of which are defined in this
file. This change adds a "FuzzTarget" struct, along with the
fuzz_add_target function, which should be used to define new fuzz
targets.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: a028edeaa6f1c154f06e16440e46b0f876a64077
      
https://github.com/qemu/qemu/commit/a028edeaa6f1c154f06e16440e46b0f876a64077
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M exec.c

  Log Message:
  -----------
  exec: keep ram block across fork when using qtest

Ram blocks were marked MADV_DONTFORK breaking fuzzing-tests which
execute each test-input in a forked process.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: d6919e4cb65230b0c8081eb072893d4e8a191a59
      
https://github.com/qemu/qemu/commit/d6919e4cb65230b0c8081eb072893d4e8a191a59
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M softmmu/vl.c

  Log Message:
  -----------
  main: keep rcu_atfork callback enabled for qtest

The qtest-based fuzzer makes use of forking to reset-state between
tests. Keep the callback enabled, so the call_rcu thread gets created
within the child process.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Acked-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: cb06fdad05f3e546a4e20f1f3c0127f9ae53de1a
      
https://github.com/qemu/qemu/commit/cb06fdad05f3e546a4e20f1f3c0127f9ae53de1a
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/fork_fuzz.c
    A tests/qtest/fuzz/fork_fuzz.h
    A tests/qtest/fuzz/fork_fuzz.ld

  Log Message:
  -----------
  fuzz: support for fork-based fuzzing.

fork() is a simple way to ensure that state does not leak in between
fuzzing runs. Unfortunately, the fuzzer mutation engine relies on
bitmaps which contain coverage information for each fuzzing run, and
these bitmaps should be copied from the child to the parent(where the
mutation occurs). These bitmaps are created through compile-time
instrumentation and they are not shared with fork()-ed processes, by
default. To address this, we create a shared memory region, adjust its
size and map it _over_ the counter region. Furthermore, libfuzzer
doesn't generally expose the globals that specify the location of the
counters/coverage bitmap. As a workaround, we rely on a custom linker
script which forces all of the bitmaps we care about to be placed in a
contiguous region, which is easy to locate and mmap over.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 275ab39d86974aab8bbce14b1a0c488653cc72d2
      
https://github.com/qemu/qemu/commit/275ab39d86974aab8bbce14b1a0c488653cc72d2
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/qos_fuzz.c
    A tests/qtest/fuzz/qos_fuzz.h

  Log Message:
  -----------
  fuzz: add support for qos-assisted fuzz targets

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: c621dc3e01c425de7da6ad82fc275e764d64e5f5
      
https://github.com/qemu/qemu/commit/c621dc3e01c425de7da6ad82fc275e764d64e5f5
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M Makefile
    M Makefile.target

  Log Message:
  -----------
  fuzz: add target/fuzz makefile rules

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: adc28027ffd9c028e42e1048385334461f65bb40
      
https://github.com/qemu/qemu/commit/adc28027ffd9c028e42e1048385334461f65bb40
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M configure

  Log Message:
  -----------
  fuzz: add configure flag --enable-fuzzing

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 04f713242d1fdb9cc03c0bff76f0750f7c8903a0
      
https://github.com/qemu/qemu/commit/04f713242d1fdb9cc03c0bff76f0750f7c8903a0
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/i440fx_fuzz.c

  Log Message:
  -----------
  fuzz: add i440fx fuzz targets

These three targets should simply fuzz reads/writes to a couple ioports,
but they mostly serve as examples of different ways to write targets.
They demonstrate using qtest and qos for fuzzing, as well as using
rebooting and forking to reset state, or not resetting it at all.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: b1db8c63169f2139af9f26c884e5e2abd27dd290
      
https://github.com/qemu/qemu/commit/b1db8c63169f2139af9f26c884e5e2abd27dd290
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/virtio_net_fuzz.c

  Log Message:
  -----------
  fuzz: add virtio-net fuzz target

The virtio-net fuzz target feeds inputs to all three virtio-net
virtqueues, and uses forking to avoid leaking state between fuzz runs.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: 472a07a6e2bd410f5679cd8a16384a6d3f474679
      
https://github.com/qemu/qemu/commit/472a07a6e2bd410f5679cd8a16384a6d3f474679
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    M tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/virtio_scsi_fuzz.c

  Log Message:
  -----------
  fuzz: add virtio-scsi fuzz target

The virtio-scsi fuzz target sets up and fuzzes the available virtio-scsi
queues. After an element is placed on a queue, the fuzzer can select
whether to perform a kick, or continue adding elements.

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: e5c59355ae9f724777c61c859292ec9db2c8c2ab
      
https://github.com/qemu/qemu/commit/e5c59355ae9f724777c61c859292ec9db2c8c2ab
  Author: Alexander Bulekov <address@hidden>
  Date:   2020-02-22 (Sat, 22 Feb 2020)

  Changed paths:
    A docs/devel/fuzzing.txt

  Log Message:
  -----------
  fuzz: add documentation to docs/devel/

Signed-off-by: Alexander Bulekov <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Darren Kenny <address@hidden>
Message-id: address@hidden
Signed-off-by: Stefan Hajnoczi <address@hidden>


  Commit: c1e667d2598b9b3ce62b8e89ed22dd38dfe9f57f
      
https://github.com/qemu/qemu/commit/c1e667d2598b9b3ce62b8e89ed22dd38dfe9f57f
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-24 (Mon, 24 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile
    M Makefile.objs
    M Makefile.target
    M block.c
    M chardev/spice.c
    M configure
    A docs/devel/fuzzing.txt
    M exec.c
    M hw/block/virtio-blk.c
    M hw/core/machine.c
    M hw/scsi/virtio-scsi.c
    M include/block/aio.h
    M include/qemu/module.h
    M include/qemu/queue.h
    M include/qemu/rcu_queue.h
    M include/sysemu/qtest.h
    M include/sysemu/sysemu.h
    M qtest.c
    M scripts/checkpatch.pl
    M scripts/get_maintainer.pl
    A softmmu/Makefile.objs
    A softmmu/main.c
    A softmmu/vl.c
    M tests/Makefile.include
    M tests/qtest/Makefile.include
    A tests/qtest/fuzz/Makefile.include
    A tests/qtest/fuzz/fork_fuzz.c
    A tests/qtest/fuzz/fork_fuzz.h
    A tests/qtest/fuzz/fork_fuzz.ld
    A tests/qtest/fuzz/fuzz.c
    A tests/qtest/fuzz/fuzz.h
    A tests/qtest/fuzz/i440fx_fuzz.c
    A tests/qtest/fuzz/qos_fuzz.c
    A tests/qtest/fuzz/qos_fuzz.h
    A tests/qtest/fuzz/virtio_net_fuzz.c
    A tests/qtest/fuzz/virtio_scsi_fuzz.c
    M tests/qtest/libqos/i2c.c
    M tests/qtest/libqos/i2c.h
    A tests/qtest/libqos/qos_external.c
    A tests/qtest/libqos/qos_external.h
    M tests/qtest/libqtest.c
    M tests/qtest/libqtest.h
    M tests/qtest/pca9552-test.c
    M tests/qtest/qos-test.c
    M tests/test-aio.c
    M tests/test-rcu-list.c
    A tests/test-rcu-slist.c
    M util/aio-posix.c
    M util/async.c
    M util/module.c
    R vl.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into 
staging

Pull request

This pull request contains a virtio-blk/scsi performance optimization, event
loop scalability improvements, and a qtest-based device fuzzing framework.  I
am including the fuzzing patches because I have reviewed them and Thomas Huth
is currently away on leave.

# gpg: Signature made Sat 22 Feb 2020 08:50:05 GMT
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <address@hidden>" [full]
# gpg:                 aka "Stefan Hajnoczi <address@hidden>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request: (31 commits)
  fuzz: add documentation to docs/devel/
  fuzz: add virtio-scsi fuzz target
  fuzz: add virtio-net fuzz target
  fuzz: add i440fx fuzz targets
  fuzz: add configure flag --enable-fuzzing
  fuzz: add target/fuzz makefile rules
  fuzz: add support for qos-assisted fuzz targets
  fuzz: support for fork-based fuzzing.
  main: keep rcu_atfork callback enabled for qtest
  exec: keep ram block across fork when using qtest
  fuzz: add fuzzer skeleton
  libqos: move useful qos-test funcs to qos_external
  libqos: split qos-test and libqos makefile vars
  libqos: rename i2c_send and i2c_recv
  qtest: add in-process incoming command handler
  libqtest: make bufwrite rely on the TransportOps
  libqtest: add a layer of abstraction to send/recv
  qtest: add qtest_server_send abstraction
  fuzz: add FUZZ_TARGET module type
  module: check module wasn't already initialized
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/88e2b97aa3e3...c1e667d2598b