Re: [PATCH 6/6] docs/migration: Add the dirty limit section

[Fabiano Rosas]

Hyman Huang <yong.huang@smartx.com> writes:

> The dirty limit feature has been introduced since the 8.1
> QEMU release but has not reflected in the document, add a
> section for that.
>
> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
> ---
>  docs/devel/migration.rst | 71 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 71 insertions(+)
>
> diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> index c3e1400c0c..1cbec22e2a 100644
> --- a/docs/devel/migration.rst
> +++ b/docs/devel/migration.rst
> @@ -588,6 +588,77 @@ path.
>       Return path  - opened by main thread, written by main thread AND 
> postcopy
>       thread (protected by rp_mutex)
>  
> +Dirty limit
> +=====================
> +The dirty limit, short for dirty page rate upper limit, is a new capability
> +introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM
> +dirty ring to throttle down the guest during live migration.
> +
> +The algorithm framework is as follows:
> +
> +::
> +
> +  
> ------------------------------------------------------------------------------
> +  main   --------------> throttle thread ------------> PREPARE(1) <--------
> +  thread  \                                                |              |
> +           \                                               |              |
> +            \                                              V              |
> +             -\                                        CALCULATE(2)       |
> +               \                                           |              |
> +                \                                          |              |
> +                 \                                         V              |
> +                  \                                    SET PENALTY(3) -----
> +                   -\                                      |
> +                     \                                     |
> +                      \                                    V
> +                       -> virtual CPU thread -------> ACCEPT PENALTY(4)
> +  
> ------------------------------------------------------------------------------
> +
> +When the qmp command qmp_set_vcpu_dirty_limit is called for the first time,
> +the QEMU main thread starts the throttle thread. The throttle thread, once
> +launched, executes the loop, which consists of three steps:
> +
> +  - PREPARE (1)
> +
> +     The entire work of PREPARE (1) is prepared for the second stage,

s/prepare/preparation/ might be more appropriate

> +     CALCULATE(2), as the name implies. It involves preparing the dirty
> +     page rate value and the corresponding upper limit of the VM:
> +     The dirty page rate is calculated via the KVM dirty ring mechanism,
> +     which tells QEMU how many dirty pages a virtual CPU has had since the
> +     last KVM_EXIT_DIRTY_RING_RULL exception; The dirty page rate upper

s/RULL/FULL

> +     limit is specified by caller, therefore fetch it directly.
> +
> +  - CALCULATE (2)
> +
> +     Calculate a suitable sleep period for each virtual CPU, which will be
> +     used to determine the penalty for the target virtual CPU. The
> +     computation must be done carefully in order to reduce the dirty page

There's a non-breaking space artifact between 'the' and 'dirty'

> +     rate progressively down to the upper limit without oscillation. To
> +     achieve this, two strategies are provided: the first is to add or
> +     subtract sleep time based on the ratio of the current dirty page rate
> +     to the limit, which is used when the current dirty page rate is far
> +     from the limit; the second is to add or subtract a fixed time when
> +     the current dirty page rate is close to the limit.
> +
> +  - SET PENALTY (3)
> +
> +     Set the sleep time for each virtual CPU that should be penalized based
> +     on the results of the calculation supplied by step CALCULATE (2).
> +
> +After completing the three above stages, the throttle thread loops back
> +to step PREPARE (1) until the dirty limit is reached.
> +
> +On the other hand, each virtual CPU thread reads the sleep duration and
> +sleeps in the path of the KVM_EXIT_DIRTY_RING_RULL exception handler, that

s/RULL/FULL

> +is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will
> +obviously exit to the path and get penalized, whereas virtual CPUs involved
> +with read processes will not.
> +
> +In summary, thanks to the KVM dirty ring technology, the dirty limit
> +algorithm will restrict virtual CPUs as needed to keep their dirty page
> +rate inside the limit. This leads to more steady reading performance during
> +live migration and can aid in improving large guest responsiveness.
> +
>  Postcopy
>  ========