Re: [PULL 0/7] Migration.next patches

[Juan Quintela]

"lizhijian@fujitsu.com" <lizhijian@fujitsu.com> wrote:
> On 10/09/2021 13:20, Li Zhijian wrote:
>>
>>
>> On 10/09/2021 00:10, Juan Quintela wrote:
>>> "Li, Zhijian" <lizhijian@cn.fujitsu.com> wrote:
>>>> on 2021/9/9 21:42, Peter Maydell wrote:
>>>>> On Thu, 9 Sept 2021 at 11:36, Juan Quintela <quintela@redhat.com> wrote:
>>>>> Fails to build, FreeBSD:
>>>>>
>>>>> ../src/migration/rdma.c:1146:23: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE'
>>>>>       int advice = wr ? IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE :
>>>>>                         ^
>>>>> ../src/migration/rdma.c:1147:18: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_ADVICE_PREFETCH'
>>>>>                    IBV_ADVISE_MR_ADVICE_PREFETCH;
>>>>>                    ^
>>>>> ../src/migration/rdma.c:1150:11: warning: implicit declaration of
>>>>> function 'ibv_advise_mr' is invalid in C99
>>>>> [-Wimplicit-function-declaration]
>>>>>       ret = ibv_advise_mr(pd, advice,
>>>>>             ^
>>>>> ../src/migration/rdma.c:1151:25: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_FLAG_FLUSH'
>>>>>                           IBV_ADVISE_MR_FLAG_FLUSH, &sg_list, 1);
>>>>>                           ^
>>>>>
>>>> it's introduced by [PULL 4/7] migration/rdma: advise prefetch write for 
>>>> ODP region
>>>> where it calls a ibv_advise_mr(). i have checked the latest FreeBSD, it 
>>>> didn't ship with this API
>>>> May i know if just FressBSD reports this failure? if so, i just need 
>>>> filtering out FreeBSD only
>>> Second try.  I can't see an example where they search for:
>>> a symbol on the header file
>>>    and
>>> a function in a library
>>>
>>> so I assume that if you have the symbols, you have the function.
>>>
>>> How do you see it?
>>>
>>> Trying to compile it on vm-build-freebsd, but not being very sucessfull
>>> so far.
>
> BTW: Does QEMU provide any mean to set http(s)_proxy to building vm ? 
> Currently, i have to
> hack the code like:
>
> -        self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
> + self.ssh_root_check("setenv HTTP_PROXY http://myproxy; setenv
> HTTPS_PROXY http://myproxy; pkg install -y %s\n" % "
> ".join(self.pkgs))

Dunno.  I don't need http proxy, for me what fails is the "tar" stage.

(master)$ make HOME=/scratch/tmp/ vm-build-fedora
    VM-BUILD fedora 
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
failed append submodule slirp to 
/mnt/code/qemu/full/vm-test-eqebvy5r.tmp/data-3a52c.tar
Failed to prepare guest environment
Traceback (most recent call last):
  File "/mnt/code/qemu/full/tests/vm/basevm.py", line 636, in main
    vm.add_source_dir(args.build_qemu)
  File "/mnt/code/qemu/full/tests/vm/basevm.py", line 270, in add_source_dir
    subprocess.check_call(["./scripts/archive-source.sh", tarfile],
  File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./scripts/archive-source.sh', 
'/mnt/code/qemu/full/vm-test-eqebvy5r.tmp/data-3a52c.tar']' returned non-zero 
exit status 1.
make: *** [tests/vm/Makefile.include:96: vm-build-fedora] Error 2

forget about the HOME= change, it also fails if I don't use it.  And the
problem is the archive bits.


>>>> on 2021/9/9 21:42, Peter Maydell wrote:
>>>>> On Thu, 9 Sept 2021 at 11:36, Juan Quintela <quintela@redhat.com> wrote:
>>>>> Fails to build, FreeBSD:
>>>>>
>>>>> ../src/migration/rdma.c:1146:23: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE'
>>>>>       int advice = wr ? IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE :
>>>>>                         ^
>>>>> ../src/migration/rdma.c:1147:18: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_ADVICE_PREFETCH'
>>>>>                    IBV_ADVISE_MR_ADVICE_PREFETCH;
>>>>>                    ^
>>>>> ../src/migration/rdma.c:1150:11: warning: implicit declaration of
>>>>> function 'ibv_advise_mr' is invalid in C99
>>>>> [-Wimplicit-function-declaration]
>>>>>       ret = ibv_advise_mr(pd, advice,
>>>>>             ^
>>>>> ../src/migration/rdma.c:1151:25: error: use of undeclared identifier
>>>>> 'IBV_ADVISE_MR_FLAG_FLUSH'
>>>>>                           IBV_ADVISE_MR_FLAG_FLUSH, &sg_list, 1);
>>>>>                           ^
>>>>>
>>>> it's introduced by [PULL 4/7] migration/rdma: advise prefetch write for 
>>>> ODP region
>>>> where it calls a ibv_advise_mr(). i have checked the latest FreeBSD, it 
>>>> didn't ship with this API
>>>> May i know if just FressBSD reports this failure? if so, i just need 
>>>> filtering out FreeBSD only
>>> Second try.  I can't see an example where they search for:
>>> a symbol on the header file
>>>    and
>>> a function in a library
>>>
>>> so I assume that if you have the symbols, you have the function.
>>>
>>> How do you see it?
>>>
>>> Trying to compile it on vm-build-freebsd, but not being very sucessfull
>>> so far.
>
> BTW: Does QEMU provide any mean to set http(s)_proxy to building vm ? 
> Currently, i have to
> hack the code like:
>
> -        self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
> + self.ssh_root_check("setenv HTTP_PROXY http://myproxy; setenv
> HTTPS_PROXY http://myproxy; pkg install -y %s\n" % "
> ".join(self.pkgs))

Dunno.  I don't need http proxy, for me what fails is the "tar" stage.

(master)$ make HOME=/scratch/tmp/ vm-build-fedora
    VM-BUILD fedora 
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
failed append submodule slirp to 
/mnt/code/qemu/full/vm-test-eqebvy5r.tmp/data-3a52c.tar
Failed to prepare guest environment
Traceback (most recent call last):
  File "/mnt/code/qemu/full/tests/vm/basevm.py", line 636, in main
    vm.add_source_dir(args.build_qemu)
  File "/mnt/code/qemu/full/tests/vm/basevm.py", line 270, in add_source_dir
    subprocess.check_call(["./scripts/archive-source.sh", tarfile],
  File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./scripts/archive-source.sh', 
'/mnt/code/qemu/full/vm-test-eqebvy5r.tmp/data-3a52c.tar']' returned non-zero 
exit status 1.
make: *** [tests/vm/Makefile.include:96: vm-build-fedora] Error 2

forget about the HOME= change, it also fails if I don't use it.  And the
problem is the archive bits.

master)$  ./scripts/archive-source.sh /tmp/kk.tar
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
failed append submodule slirp to /tmp/kk.tar

See that the problem is the slirp submodule, but it has the "right"
version, i.e. not a case of updating the module.  the dtc works without problem.

(master)$ sh -x ./scripts/archive-source.sh /tmp/kk.tar
+ test 1 -lt 1
++ realpath /tmp/kk.tar
+ tar_file=/tmp/kk.tar
++ mktemp -d /tmp/kk.sub.XXXXXXXX
+ sub_tdir=/tmp/kk.sub.WKj1o6oP
+ sub_file=/tmp/kk.sub.WKj1o6oP/submodule.tar
+ submodules='dtc slirp meson ui/keycodemapdb'
+ submodules='dtc slirp meson ui/keycodemapdb tests/fp/berkeley-softfloat-3 
tests/fp/berkeley-testfloat-3'
+ sub_deinit=
+ trap cleanup 0 1 2 3 15
++ tree_ish
++ local retval=HEAD
++ git diff-index --quiet --ignore-submodules=all HEAD --
++ echo HEAD
+ git archive --format tar HEAD
+ test 0 -ne 0
+ for sm in $submodules
++ git submodule status dtc
+ status=' 85e5d839847af54efab170f2b1331b2a6421e647 dtc (v1.6.0-4-g85e5d83)'
+ smhash='85e5d839847af54efab170f2b1331b2a6421e647 dtc (v1.6.0-4-g85e5d83)'
+ smhash=85e5d839847af54efab170f2b1331b2a6421e647
+ case "$status" in
+ cd dtc
++ tree_ish
++ local retval=HEAD
++ git diff-index --quiet --ignore-submodules=all HEAD --
++ echo HEAD
+ git archive --format tar --prefix dtc/ HEAD
+ test 0 -ne 0
+ tar --concatenate --file /tmp/kk.tar /tmp/kk.sub.WKj1o6oP/submodule.tar
+ test 0 -ne 0
+ for sm in $submodules
++ git submodule status slirp
+ status=' a88d9ace234a24ce1c17189642ef9104799425e0 slirp (v4.6.1-7-ga88d9ac)'
+ smhash='a88d9ace234a24ce1c17189642ef9104799425e0 slirp (v4.6.1-7-ga88d9ac)'
+ smhash=a88d9ace234a24ce1c17189642ef9104799425e0
+ case "$status" in
+ cd slirp
++ tree_ish
++ local retval=HEAD
++ git diff-index --quiet --ignore-submodules=all HEAD --
++ echo HEAD
+ git archive --format tar --prefix slirp/ HEAD
+ test 0 -ne 0
+ tar --concatenate --file /tmp/kk.tar /tmp/kk.sub.WKj1o6oP/submodule.tar
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
+ test 2 -ne 0
+ error 'failed append submodule slirp to /tmp/kk.tar'
+ printf '%s\n' 'failed append submodule slirp to /tmp/kk.tar'
failed append submodule slirp to /tmp/kk.tar
+ exit 1
+ cleanup
+ local status=1
+ rm -rf /tmp/kk.sub.WKj1o6oP
+ test '' '!=' ''
+ exit 1
(master)$ 

Doing the things on the command line, the 

  git archive --format tar --prefix slirp/ HEAD

Creates a tar archive, so I get completely lost.

I showed here fedora, but it fails exactly the same for freebsd,
openbsd, ... and everything that I decided to build.  It fails in the
smae stage.

Later, Juan.