[Bug 1833204] Re: VM failed to start in nested virtualization with error "KVM: entry failed, hardware error 0x0"

[Thomas Huth]

The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid
and which could be closed already. Thus we are setting older bugs to
"Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.


** Changed in: qemu
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1833204

Title:
  VM failed to start in nested virtualization with error "KVM: entry
  failed, hardware error 0x0"

Status in QEMU:
  Incomplete

Bug description:
  Hi,

  I have 3 ubuntu nodes provisioned by IaaS. 
  Then I tried to launch VM again in my ubuntu nodes.
  It's a little strange that VM could be started successfully in two nodes. 
  And always failed in one nodes with error "KVM: entry failed, hardware error 
0x0". 

  When using virsh to resume the VM, it failed with following error,
  virsh # list
   Id   Name                State
  ----------------------------------
   1    default_vm-cirros   paused

  virsh # resume default_vm-cirros
  error: Failed to resume domain default_vm-cirros
  error: internal error: unable to execute QEMU command 'cont': Resetting the 
Virtual Machine is required

  
  The detailed log from /var/log/libvirt/qemu/default_vm-cirros.log is as below.
  ```
  2019-06-18 09:55:52.397+0000: starting up libvirt version: 5.0.0, package: 
1.fc28 (Unknown, 2019-01-22-08:04:34, 64723eea657e48d296e6beb0b1be9c4c), qemu 
version: 3.1.0qemu-3.1.0-4.fc28, kernel: 4.15.0-47-generic, hostname: vm-cirros
  LC_ALL=C \
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  HOME=/root \
  QEMU_AUDIO_DRV=none \
  /usr/bin/qemu-system-x86_64 \
  -name guest=default_vm-cirros,debug-threads=on \
  -S \
  -object 
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-default_vm-cirros/master-key.aes
 \
  -machine pc-q35-3.1,accel=kvm,usb=off,dump-guest-core=off \
  -cpu 
Broadwell-IBRS,vme=on,ss=on,vmx=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc_adjust=on,mpx=on,avx512f=on,avx512cd=on,ssbd=on,xsaveopt=on,abm=on,invpcid=off
 \
  -m 489 \
  -realtime mlock=off \
  -smp 1,sockets=1,cores=1,threads=1 \
  -object iothread,id=iothread1 \
  -uuid 0d2a2043-41c0-59c3-9b17-025022203668 \
  -no-user-config \
  -nodefaults \
  -chardev socket,id=charmonitor,fd=22,server,nowait \
  -mon chardev=charmonitor,id=monitor,mode=control \
  -rtc base=utc \
  -no-shutdown \
  -boot strict=on \
  -device 
pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2
 \
  -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
  -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
  -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
  -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
  -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x0 \
  -drive 
file=/var/run/kubevirt-ephemeral-disks/container-disk-data/default/vm-cirros/disk_containerdisk/disk-image.raw,format=raw,if=none,id=drive-ua-containerdisk,cache=none
 \
  -device 
virtio-blk-pci,scsi=off,bus=pci.3,addr=0x0,drive=drive-ua-containerdisk,id=ua-containerdisk,bootindex=1,write-cache=on
 \
  -drive 
file=/var/run/kubevirt-ephemeral-disks/cloud-init-data/default/vm-cirros/noCloud.iso,format=raw,if=none,id=drive-ua-cloudinitdisk,cache=none
 \
  -device 
virtio-blk-pci,scsi=off,bus=pci.4,addr=0x0,drive=drive-ua-cloudinitdisk,id=ua-cloudinitdisk,write-cache=on
 \
  -netdev tap,fd=24,id=hostua-default,vhost=on,vhostfd=25 \
  -device 
virtio-net-pci,host_mtu=1430,netdev=hostua-default,id=ua-default,mac=16:57:38:cd:57:cb,bus=pci.1,addr=0x0
 \
  -chardev socket,id=charserial0,fd=26,server,nowait \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -chardev socket,id=charchannel0,fd=27,server,nowait \
  -device 
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
 \
  -vnc 
vnc=unix:/var/run/kubevirt-private/3b22a138-91af-11e9-af36-0016ac101123/virt-vnc
 \
  -device VGA,id=video0,vgamem_mb=16,bus=pcie.0,addr=0x1 \
  -sandbox 
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
  -msg timestamp=on
  KVM: entry failed, hardware error 0x0
  EAX=00000000 EBX=00000000 ECX=00000000 EDX=000306d2
  ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
  EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
  ES =0000 00000000 0000ffff 00009300
  CS =f000 ffff0000 0000ffff 00009b00
  SS =0000 00000000 0000ffff 00009300
  DS =0000 00000000 0000ffff 00009300
  FS =0000 00000000 0000ffff 00009300
  GS =0000 00000000 0000ffff 00009300
  LDT=0000 00000000 0000ffff 00008200
  TR =0000 00000000 0000ffff 00008b00
  GDT=     00000000 0000ffff
  IDT=     00000000 0000ffff
  CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
  DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 
DR3=0000000000000000 
  DR6=00000000ffff0ff0 DR7=0000000000000400
  EFER=0000000000000000
  Code=06 66 05 00 00 01 00 8e c1 26 66 a3 14 f5 66 5b 66 5e 66 c3 <ea> 5b e0 
00 f0 30 36 2f 32 33 2f 39 39 00 fc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  ```

  Ubuntu node version as follow,
  cat /etc/os-release 
  NAME="Ubuntu"
  VERSION="18.04.2 LTS (Bionic Beaver)"
  ID=ubuntu
  ID_LIKE=debian
  PRETTY_NAME="Ubuntu 18.04.2 LTS"
  VERSION_ID="18.04"
  HOME_URL="https://www.ubuntu.com/";
  SUPPORT_URL="https://help.ubuntu.com/";
  BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/";
  
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy";
  VERSION_CODENAME=bionic
  UBUNTU_CODENAME=bionic

  Output of `uname -a` is:
  4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64 x86_64 
x86_64 GNU/Linux


  Any additional information needed, please let me know.
  Thx.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1833204/+subscriptions