Re: [PATCH v7 07/12] multifd: Prepare to send a packet without the mutex held

[Juan Quintela]

Leonardo Brás <leobras@redhat.com> wrote:
> On Tue, 2022-08-02 at 08:39 +0200, Juan Quintela wrote:
>> We do the send_prepare() and the fill of the head packet without the
>> mutex held.  It will help a lot for compression and later in the
>> series for zero pages.
>> 
>> Notice that we can use p->pages without holding p->mutex because
>> p->pending_job == 1.
>> 
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  migration/multifd.h |  2 ++
>>  migration/multifd.c | 11 ++++++-----
>>  2 files changed, 8 insertions(+), 5 deletions(-)
>> 
>> diff --git a/migration/multifd.h b/migration/multifd.h
>> index a67cefc0a2..cd389d18d2 100644
>> --- a/migration/multifd.h
>> +++ b/migration/multifd.h
>> @@ -109,7 +109,9 @@ typedef struct {
>>      /* array of pages to sent.
>>       * The owner of 'pages' depends of 'pending_job' value:
>>       * pending_job == 0 -> migration_thread can use it.
>> +     *                     No need for mutex lock.
>>       * pending_job != 0 -> multifd_channel can use it.
>> +     *                     No need for mutex lock.
>>       */
>>      MultiFDPages_t *pages;
>>  
>> diff --git a/migration/multifd.c b/migration/multifd.c
>> index 09a40a9135..68fc9f8e88 100644
>> --- a/migration/multifd.c
>> +++ b/migration/multifd.c
>> @@ -663,6 +663,8 @@ static void *multifd_send_thread(void *opaque)
>>                  p->flags |= MULTIFD_FLAG_SYNC;
>>                  p->sync_needed = false;
>>              }
>> +            qemu_mutex_unlock(&p->mutex);
>> +
>
> If it unlocks here, we will have unprotected:
> for (int i = 0; i < p->pages->num; i++) {
>     p->normal[p->normal_num] = p->pages->offset[i];
>     p->normal_num++;
> }
>
> And p->pages seems to be in the mutex-protected area.
> Should it be ok?

>From the documentation:

    /* array of pages to sent.
     * The owner of 'pages' depends of 'pending_job' value:
     * pending_job == 0 -> migration_thread can use it.
     *                     No need for mutex lock.
     * pending_job != 0 -> multifd_channel can use it.
     *                     No need for mutex lock.
     */
    MultiFDPages_t *pages;

So, it is right.

> Also, under that we have:
>             if (p->normal_num) {
>                 ret = multifd_send_state->ops->send_prepare(p, &local_err);
>                 if (ret != 0) {
>                     qemu_mutex_unlock(&p->mutex);
>                     break;
>                 }
>             }
>
> Calling mutex_unlock() here, even though the unlock already happened before,
> could cause any issue?

Good catch.  Never got an error there.

Removing that bit.

> Best regards,


Thanks, Juan.