[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: tools/virtiofs: Multi threading seems to hurt performance
From: |
Stefan Hajnoczi |
Subject: |
Re: tools/virtiofs: Multi threading seems to hurt performance |
Date: |
Mon, 21 Sep 2020 09:39:23 +0100 |
On Fri, Sep 18, 2020 at 05:34:36PM -0400, Vivek Goyal wrote:
> And here are the comparision results. To me it seems that by default
> we should switch to 1 thread (Till we can figure out how to make
> multi thread performance better even when single process is doing
> I/O in client).
Let's understand the reason before making changes.
Questions:
* Is "1-thread" --thread-pool-size=1?
* Was DAX enabled?
* How does cache=none perform?
* Does commenting out vu_queue_get_avail_bytes() + fuse_log("%s:
Queue %d gave evalue: %zx available: in: %u out: %u\n") in
fv_queue_thread help?
* How do the kvm_stat vmexit counters compare?
* How does host mpstat -P ALL compare?
* How does host perf record -a compare?
* Does the Rust virtiofsd show the same pattern (it doesn't use glib
thread pools)?
Stefan
> NAME WORKLOAD Bandwidth IOPS
>
> cache-auto seqread-psync 690(MiB/s) 172k
>
> cache-auto-1-thread seqread-psync 729(MiB/s) 182k
>
>
> cache-auto seqread-psync-multi 2578(MiB/s) 644k
>
> cache-auto-1-thread seqread-psync-multi 2597(MiB/s) 649k
>
>
> cache-auto seqread-mmap 660(MiB/s) 165k
>
> cache-auto-1-thread seqread-mmap 672(MiB/s) 168k
>
>
> cache-auto seqread-mmap-multi 2499(MiB/s) 624k
>
> cache-auto-1-thread seqread-mmap-multi 2618(MiB/s) 654k
>
>
> cache-auto seqread-libaio 286(MiB/s) 71k
>
> cache-auto-1-thread seqread-libaio 260(MiB/s) 65k
>
>
> cache-auto seqread-libaio-multi 1508(MiB/s) 377k
>
> cache-auto-1-thread seqread-libaio-multi 986(MiB/s) 246k
>
>
> cache-auto randread-psync 35(MiB/s) 9191
>
> cache-auto-1-thread randread-psync 55(MiB/s) 13k
>
>
> cache-auto randread-psync-multi 179(MiB/s) 44k
>
> cache-auto-1-thread randread-psync-multi 209(MiB/s) 52k
>
>
> cache-auto randread-mmap 32(MiB/s) 8273
>
> cache-auto-1-thread randread-mmap 50(MiB/s) 12k
>
>
> cache-auto randread-mmap-multi 161(MiB/s) 40k
>
> cache-auto-1-thread randread-mmap-multi 185(MiB/s) 46k
>
>
> cache-auto randread-libaio 268(MiB/s) 67k
>
> cache-auto-1-thread randread-libaio 254(MiB/s) 63k
>
>
> cache-auto randread-libaio-multi 256(MiB/s) 64k
>
> cache-auto-1-thread randread-libaio-multi 155(MiB/s) 38k
>
>
> cache-auto seqwrite-psync 23(MiB/s) 6026
>
> cache-auto-1-thread seqwrite-psync 30(MiB/s) 7925
>
>
> cache-auto seqwrite-psync-multi 100(MiB/s) 25k
>
> cache-auto-1-thread seqwrite-psync-multi 154(MiB/s) 38k
>
>
> cache-auto seqwrite-mmap 343(MiB/s) 85k
>
> cache-auto-1-thread seqwrite-mmap 355(MiB/s) 88k
>
>
> cache-auto seqwrite-mmap-multi 408(MiB/s) 102k
>
> cache-auto-1-thread seqwrite-mmap-multi 438(MiB/s) 109k
>
>
> cache-auto seqwrite-libaio 41(MiB/s) 10k
>
> cache-auto-1-thread seqwrite-libaio 65(MiB/s) 16k
>
>
> cache-auto seqwrite-libaio-multi 137(MiB/s) 34k
>
> cache-auto-1-thread seqwrite-libaio-multi 214(MiB/s) 53k
>
>
> cache-auto randwrite-psync 22(MiB/s) 5801
>
> cache-auto-1-thread randwrite-psync 30(MiB/s) 7927
>
>
> cache-auto randwrite-psync-multi 100(MiB/s) 25k
>
> cache-auto-1-thread randwrite-psync-multi 151(MiB/s) 37k
>
>
> cache-auto randwrite-mmap 31(MiB/s) 7984
>
> cache-auto-1-thread randwrite-mmap 55(MiB/s) 13k
>
>
> cache-auto randwrite-mmap-multi 124(MiB/s) 31k
>
> cache-auto-1-thread randwrite-mmap-multi 213(MiB/s) 53k
>
>
> cache-auto randwrite-libaio 40(MiB/s) 10k
>
> cache-auto-1-thread randwrite-libaio 64(MiB/s) 16k
>
>
> cache-auto randwrite-libaio-multi 139(MiB/s) 34k
>
> cache-auto-1-thread randwrite-libaio-multi 212(MiB/s) 53k
>
>
>
>
>
>
>
signature.asc
Description: PGP signature