[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 7/9] tests/performance: Add nightly tests
From: |
Ahmed Karaman |
Subject: |
Re: [PATCH 7/9] tests/performance: Add nightly tests |
Date: |
Wed, 2 Sep 2020 19:29:19 +0200 |
Thanks Mr. Alex,
On Wed, Sep 2, 2020 at 3:26 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Ahmed Karaman <ahmedkhaledkaraman@gmail.com> writes:
>
> > A nightly performance testing system to monitor any change in QEMU
> > performance across seventeen different targets.
> >
> > The system includes eight different benchmarks to provide a variety
> > of testing workloads.
> >
> > dijkstra_double:
> > Find the shortest path between the source node and all other nodes
> > using Dijkstra’s algorithm. The graph contains n nodes where all nxn
> > distances are double values. The value of n can be specified using
> > the -n flag. The default value is 2000.
> >
> > dijkstra_int32:
> > Find the shortest path between the source node and all other nodes
> > using Dijkstra’s algorithm. The graph contains n nodes where all nxn
> > distances are int32 values. The value of n can be specified using
> > the -n flag. The default value is 2000.
> >
> > matmult_double:
> > Standard matrix multiplication of an n*n matrix of randomly generated
> > double numbers from 0 to 100. The value of n is passed as an argument
> > with the -n flag. The default value is 200.
> >
> > matmult_int32:
> > Standard matrix multiplication of an n*n matrix of randomly generated
> > integer numbers from 0 to 100. The value of n is passed as an
> > argument with the -n flag. The default value is 200.
> >
> > qsort_double:
> > Quick sort of an array of n randomly generated double numbers from 0
> > to 1000. The value of n is passed as an argument with the -n flag.
> > The default value is 300000.
> >
> > qsort_int32:
> > Quick sort of an array of n randomly generated integer numbers from 0
> > to 50000000. The value of n is passed as an argument with the -n
> > flag.The default value is 300000.
> >
> > qsort_string:
> > Quick sort of an array of 10000 randomly generated strings of size 8
> > (including null terminating character). The sort process is repeated
> > n number of times. The value of n is passed as an argument with the
> > -n flag. The default value is 20.
> >
> > search_string:
> > Search for the occurrence of a small string in a much larger random
> > string (“needle in a hay”). The search process is repeated n number
> > of times and each time, a different large random string (“hay”) is
> > generated. The value of n can be specified using the -n flag. The
> > default value is 20.
> >
> > Syntax:
> > nightly_tests_core.py [-h] [-r REF]
> > Optional arguments:
> > -h, --help Show this help message and exit
> > -r REF, --reference REF
> > Reference QEMU version - Default is v5.1.0
> > Example of usage:
> > nightly_tests_core.py -r v5.1.0 2>log.txt
> >
> > The following report includes detailed setup and execution details
> > of the system:
> > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/
> >
> > Signed-off-by: Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
> > ---
> > tests/performance/nightly-tests/README.md | 243 +++++
> > .../source/dijkstra_double/dijkstra_double.c | 194 ++++
> > .../source/dijkstra_int32/dijkstra_int32.c | 192 ++++
> > .../source/matmult_double/matmult_double.c | 123 +++
> > .../source/matmult_int32/matmult_int32.c | 121 +++
> > .../source/qsort_double/qsort_double.c | 104 ++
> > .../source/qsort_int32/qsort_int32.c | 103 ++
> > .../source/qsort_string/qsort_string.c | 122 +++
> > .../source/search_string/search_string.c | 110 +++
> > .../scripts/nightly_tests_core.py | 920 ++++++++++++++++++
> > .../scripts/run_nightly_tests.py | 135 +++
> > .../nightly-tests/scripts/send_email.py | 56 ++
> > 12 files changed, 2423 insertions(+)
> > create mode 100644 tests/performance/nightly-tests/README.md
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/dijkstra_double/dijkstra_double.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/dijkstra_int32/dijkstra_int32.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/matmult_double/matmult_double.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/matmult_int32/matmult_int32.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/qsort_double/qsort_double.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/qsort_int32/qsort_int32.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/qsort_string/qsort_string.c
> > create mode 100644
> > tests/performance/nightly-tests/benchmarks/source/search_string/search_string.c
>
> Perhaps we could compress these paths down to:
>
> tests/tcg/benchmarks/foo.c
> tests/tcg/benchmarks/bar.c
>
> and then we can also ensure they are built using the existing TCG tests
> cross compile framework.
>
Can you provide the commands for compiling the benchmarks for the 17 targets?
> > create mode 100755
> > tests/performance/nightly-tests/scripts/nightly_tests_core.py
> > create mode 100755
> > tests/performance/nightly-tests/scripts/run_nightly_tests.py
> > create mode 100644 tests/performance/nightly-tests/scripts/send_email.py
> >
> > diff --git a/tests/performance/nightly-tests/README.md
> > b/tests/performance/nightly-tests/README.md
> > new file mode 100644
> > index 0000000000..6db3b351b3
> > --- /dev/null
> > +++ b/tests/performance/nightly-tests/README.md
> > @@ -0,0 +1,243 @@
> > +### QEMU Nightly Tests
> > +
> > +**Required settings:**
> > +
> > +Update the `GMAIL_USER` object in `send_email.py` with your credentials.
> > +
> > +For more details on how the system works, please check the [eighth
> > report](https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/)
> > of the "TCG Continuos Benchmarking" series.
>
> As external URLs are potentially unstable I think we want to distil the
> details into a rst do in docs/devel/
Any advice on converting markdown to rst? Are there any specific rules
that should be followed to write rst documentation for QEMU.
>
> <snip>
>
> --
> Alex Bennée
Best regards,
Ahmed Karaman