-
Notifications
You must be signed in to change notification settings - Fork 30
Tests
| Case ID | Domain | Case Name | Test Purpose |
|---|---|---|---|
| 1 | sanity | tdx_enabled | Check if TDX is enabled in host and guest |
| 2 | sanity | tdx_VMX_coexist | Check whether TDX and VMX guests can co-exist. |
| 3 | sanity | tdx_NOEPT | Check when EPT=no, TDX guest should not be created. |
| 4 | sanity | tdx_tsc_config | Check TSC function in TDX guest |
| 5 | sanity | tdx_debugon_nmi | Check whether TDX guest can boot up with debug=on, and run "nmi" in qemu monitor. |
| 6 | sanity | tdx_debugoff_nmi | Check whether TDX guest can boot up with debug=off, and run "nmi" in qemu monitor. |
| 7 | sanity | tdx_huge_resource_VM | Check whether one TDX guest with half cpu and half memory of the platform |
| 8 | sanity | tdx_1vcpus_10times | Check whether 1 vcpu TDX guest can be created 10 times continuously |
| 9 | sanity | tdx_4vcpus_1socket_10times | Check whether 4 vcpu and 1 socket TDX guest can be created 10 times continuously |
| 10 | sanity | tdx_4vcpus_2sockets_5times | Check whether 4 vcpu and 2 socks TDX guest can be created 5 times continuously |
| 11 | sanity | tdx_2TD_VM | Check whether 2 TDX guest can be created together |
| 12 | tdx | tdx_disabled | Check if TDX can be disabled on host and guest |
| 13 | tdx | tdx_8vcpus | Check if TDX guest can boot up with 8 vcpus |
| 14 | tdx | tdx_NO_TDVF | Check if TDX guest can boot up with no TDVF |
| 15 | tdx | tdx_build_kernel | Build kernel on TDX guest |
| 16 | tdx | tdx_debug_on | Check whether TDX guest can boot up with debug=on |
| 17 | tdx | tdx_debug_off | Check whether TDX guest can boot up with debug=off |
| 18 | tdx | tdx_debugoff_earlyprink | Check whether TDX guest can boot up with debug=off and "earlyprintk" appended in guest kernel command line |
| 19 | tdx | tdx_hdevents_in5sec | Collect hardware events via perf during 5 seconds and check "Performance counter stats" |
| 20 | tdx | tdx_VMP_cpu_onoff | Check TDX guest works when some host cpu off/on |
| 21 | tdx | tdx_31guests | Check whether 31 TD guests can bootup together |
| 22 | tdx | tdx_16G_4guests_20times | Check whether 4 TDX guests with 16G memory can be created, repeat 20 times |
| 23 | tdx | tdx_uefi | Check whether TDX SEAM modules are loaded by UEFI |
| 24 | tdx | tdx_cpuoff_pinedVMdown | Boot TD guest with 64 vcpu, pin the qemu process to a host cpu, offline the host cpu, no crash on guest and host, kill the VM, repeat 20 times. |
| 25 | tdx | tdx_tsc_deadline_enable | Check if TSC deadline are enabled on TD guests by default |
| 26 | tdx | tdx_tsc_deadline_disable | Check if TSC deadline can be disabled on TD guests by qemu command |
| 27 | tdx | tdx_vsock_vm_server | Check iperf works with vsock: guest as server and host as client |
| 28 | tdx | tdx_vsock_vm_client | Check iperf works with vsock: guest as client and host as server |
Guest network interface
- For host without bridge, we use qemu hostfwd(default in test case)
Login to VM: ssh -p 10022 user@localhost-device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22
- For host with bridge(need to setup bridge first)
-device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on
Don't use the same MAC address for multiple VMs, which will cause MAC/IP conflict
VM can get a normal IP address by cridge, you can check the ip in VM via GUI(vnc) or stdio
Or get IP by mac using arp-scan: arp-scan -l -I virbr0|grep "00:b8:8b:f1:04:56"
Login to VM: ssh $guest_ip
Guest console in stdio
-chardev stdio,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux
Login to VM directly in stdio
Run guest in daemonize(default in test cases)
-chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -daemonize
Note: Use qemu hostfwd for guest network interface, run guest in daemonize
Test Purpose: Check if TDX is enabled in host and guest.
Test Steps:
- Boot host and do the following check.
(1) rdmsr 0xfe --bitfield 15:15 → to check platform supports TDX-SEAM
The expected calue is 1
Note： If get the following error, run "modprobe msr"
rdmsr: open: No such file or directory
(2) grep Y /sys/module/kvm_intel/parameters/tdx --------- to check if TDX is supported in KVM
Note: If the value is "N", do "rmmod kvm_intel; modprobe kvm_intel tdx=1" and check again
(3) Check TDX in dmesg
[root@spr-2s5 ~]# dmesg | grep -E "tdx: TDX module|tdx: SEAMRR enabled"
[ 97.578273] tdx: TDX module: atributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20230206, build_num 457
[ 98.522910] tdx: TDX module initialized.
[root@spr-2s5 ~]# dmesg |grep -E "tdx: Successfully initialized TDX module|tdx: TDX module initialized|tdx: module initialized"
[ 98.522910] tdx: TDX module initialized. - Create VM
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - After VM boots up, check tdx_guest on VM.
(1) cat /proc/cpuinfo |grep tdx_guest
(2) ls /dev/tdx_guest
Expected Result: TDX feature is enabled in host and guest
Test Purpose: Check whether TDX and VMX guests can co-exist.
Test Steps:
- Boot host and check TDX flag.
[root@spr-2s5 ~]# dmesg | grep -E "tdx: TDX module|tdx: SEAMRR enabled"
[ 97.578273] tdx: TDX module: atributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20230206, build_num 457
[ 98.522910] tdx: TDX module initialized.
[root@spr-2s5 ~]# dmesg |grep -E "tdx: Successfully initialized TDX module|tdx: TDX module initialized|tdx: module initialized"
[ 98.522910] tdx: TDX module initialized. - Create TDX guest.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10023-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Create legacy guest.
qemu-system-x86_64 -accel kvm -cpu host -drive file=centos-9-embargo_intel_next_2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0,bootindex=0 -m 4096 -monitor pty -smp 1 -cpu host -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -daemonize -bios /usr/local/share/qemu/OVMF.fd
Expected Result: These two guests can boot up successfully.
Test Purpose: Check when EPT=no, TDX guest should not be created.
Test Steps:
- Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
- Disable EPT in KVM.
(1) rmmod kvm_intel
(2) modprobe kvm_intel tdx=1 pt_mode=1 ept=0 - Create TDX guest.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Get the error information.
(1) We get the following qemu output.
qemu-system-x86_64: -accel kvm: vm-type X86_TDX_VM not supported by KVM
(2) We can get more detailed information in dmesg.
dmesg | grep TDX
...
[ 2992.482851] TDX requires mmio caching. Please enable mmio caching for TDX.
Expected Result: Fail to create TDX guest when EPT is disabled
Test Purpose: Check TSC function in TDX guest
Test Steps:
-
Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. -
Check TSC value on host
## dmesg | grep -i tsc
[ 0.000000] tsc: Detected 1000.000 MHz processor -
Check TSC value on host
(1) Check leaf 0x15 (which is related to TSC)
# cpuid -rl 0x15 -1
CPU 0:
0x00000015 0x00: eax=0x00000001 ebx=0x00000028 ecx=0x017d7840 edx=0x00000000
EAX Fixed 0x1 → 1; EDX Fixed 0x0 → 0x0; ECX Fixed 0x017D7840 → 0x017D7840; EBX As Configured: This MSR is set by TSC_FREQUENCY in TD-param and can be modified by QEMU tsc-frequency= value. (2) Calculate TSC value execute "cpuid | less" and search "Time Stamp Counter" to get below information Time Stamp Counter/Core Crystal Clock Information (0x15):
TSC/clock ratio = 40/1 (40/1 ↔ cpuid EBX/EAX)
nominal core crystal clock = 25000000 Hz (25000000Hz ↔ ecx: 0x017d7840) Host tsc frequency = ECX * EBX / EAX
-
Create TDX guest
qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -bios /usr/local/share/qemu/OVMF.fd -
Check TSC value on guest
(1) Check leaf 0x15 (which is related to TSC)
# cpuid -rl 0x15 -1
CPU 0:
0x00000015 0x00: eax=0x00000001 ebx=0x00000028 ecx=0x017d7840 edx=0x00000000
EAX Fixed 0x1 → 1; EDX Fixed 0x0 → 0x0; ECX Fixed 0x017D7840 → 0x017D7840; Guest tsc frequency = ECX * EBX / EAX Guest should have the same Guest tsc frequency as the host.
-
Re-create TDX guest with tsc-frequency=3000000000
qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -nodefaults -daemonize -cpu host,tsc-freq=3000000000 -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10023-:22 -bios /usr/local/share/qemu/OVMF.fd -
Check TSC value on guest
# dmesg | grep -i tsc
[ 0.000000] tsc: Detected 3000.000 MHz processor
# cpuid | grep -i TSC
TSC: time stamp counter = true
IA32_TSC_ADJUST MSR supported = false
TSC/clock ratio = 120/1
TSC frequency (Hz) = 3000000
RDTSCP = true
TscInvariant = false
# cpuid -rl 0x15 -1
CPU:
0x00000015 0x00: eax=0x00000001 ebx=0x00000078 ecx=0x017d7840 edx=0x00000000
Guest tsc frequency = ECX * EBX / EAX
Guest tsc frequency should equal to the configured value 3000000000
Expected Result: Without tsc-frequency set, guest should have the same TSC value as the host; when set tsc-frequency, TSC value can be configured successfully for TDX guest.
Test Purpose: Check whether TDX guest can boot up with debug=on, and run "nmi" in qemu monitor.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with -object,tdx-guest,debug=on
qemu-system-x86_64 -accel kvm -monitor pty -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -bios /usr/local/share/qemu/OVMF.fd -monitor telnet:127.0.0.1:45454,nowait,server - Login qemu monitor, and run "nmi"
[root@emr-120485 ~]# telnet 127.0.0.1 45454
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
QEMU 8.0.0 monitor - type 'help' for more information
(qemu) nmi
Note: ctrl+] to quit qemu monitor, enter "quit" in qemu monitor is to kill qemu process(kill vm)
Expected Result: TDX guest works with "nmi" in qemu monitor with debug=on
Test Purpose: Check whether TDX guest can boot up with debug=off, and run "nmi" in qemu monitor.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with -object,tdx-guest,debug=off.
qemu-system-x86_64 -accel kvm -monitor pty -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=off -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -bios /usr/local/share/qemu/OVMF.fd -monitor telnet:127.0.0.1:45454,nowait,server - Login qemu monitor, and run "nmi"
[root@emr-120485 ~]# telnet 127.0.0.1 45454
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
QEMU 8.0.0 monitor - type 'help' for more information
(qemu) nmi
Note: ctrl+] to quit qemu monitor, enter "quit" in qemu monitor is to kill qemu process(kill vm)
Expected Result: TDX guest works with "nmi" in qemu monitor with debug=off
Test Purpose: Check whether one TDX guest with half cpu and half memory of the platform
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Check cpu number and memory of the platfrom
grep -c processor /proc/cpuinfo
free -h | grep Mem | awk '{print $4}' | awk -FG '{print $1}' - Create TDX guest with half memory and half cpu numbers of the platform.
qemu-system-x86_64 -accel kvm -cpu host -smp 112 -m 121G -object tdx-guest,id=tdx -machine q35,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=121G,private=on -nographic -vga none -drive file=/share/xvs/var/tmp-img_tdx_huge_resource_VM_1690940795,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -bios /usr/local/share/qemu/OVMF.fd -chardev stdio,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -monitor chardev:mux
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Test Purpose: Check whether 1 vcpu TDX guest can be created 10 times continuously
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with 1 vcpu.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Destroy guest
- Repeat step1 and step2 10 times
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Test Purpose: Check whether 4 vcpu and 1 socket TDX guest can be created 10 times continuously
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with 4 vcpu 1 socket
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4,sockets=1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Destroy guest
- Repeat step1 and step2 10 times
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Test Purpose: Check whether 4 vcpu and 2 socks TDX guest can be created 5 times continuously
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with 4 vcpu and 2 socks
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4,sockets=2 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Destroy guest
- Repeat step1 and step2 5 times
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Test Purpose: Check whether 2 TDX guest can be created together
Test Steps:
-
Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. -
Create 2 TDX guests on one host
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fdqemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next_2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10023-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Case ID: 012 tdx_disabled
Test Purpose: Check if TDX can be disabled on host and guest.
Test Steps:
- Boot host and do the following operations.
(1) reload kvm_intel with tdx=0
rmmod kvm_intel
modprobe kvm_intel tdx=0
(2) rdmsr 0xfe --bitfield 15:15 → to check platform supports TDX-SEAM, it doesn’t ensure TDX-SEAM is not loaded and configured
1
(3) cat /sys/module/kvm_intel/parameters/tdx N - Create TDX guest
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Get error information
Expected Result: When TDX is disabled in host, we will fail to create td guest
Case ID: 013 tdx_8vcpus
Test Purpose: Check if TDX guest can boot up with 8 vcpus.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest with 8 vcpus.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
Expected Result: TDX guest with 8 vcpus can boot up successfully.
Case ID: 014 tdx_NO_TDVF
Test Purpose: Check if TDX guest can boot up with no TDVF.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TDX guest without OVMF.fd
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize - Get error: memory_region_init_ram_gmem: error creating gmem: Invalid argument
Expected Result: TDX guest cannot boot up without TDVF
Case ID: 015 TDX_build_kernel
Test Purpose: Build kernel on TDX guest
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. - Create TDX guest.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Login guest and build kernel
Expected Result: Kernel can be built successfully.
Case ID: 016 TDX_debug_on
Test Purpose: Check whether TDX guest can boot up with debug=on
Test Steps:
- Boot host and check TDX flag.
# dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. - Create TDX guest with -object,tdx-guest,debug=on
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
Expected Result: TDX guest can boot up successfully.
Case ID: 017 TDX_debug_off
Test Purpose: Check whether TDX guest can boot up with debug=off
Test Steps:
- Boot host and check TDX flag.
# dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. - Create TDX guest with -object,tdx-guest,debug=off
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=off -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
Expected Result: TDX guest can boot up successfully.
Case ID: 018 TDX_debugoff_earlyprink
Test Purpose: Check whether TDX guest can boot up with debug=off and "earlyprintk" appended in guest kernel command line.
Test Steps:
- Boot host and check TDX flag.
# dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. - Create TDX guest with -object,tdx-guest,debug=off, and "earlyprintk" appended in guest kernel command line, for example "earlyprintk=ttyS0,115200"
qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -nodefaults -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=off -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -kernel /boot/kvm-vmlinuz -initrd /boot/initramfs-kvm.img -append 'root=UUID=5cf6ccb0-3db3-42d2-921e-35cdd67a69bb ro console=hvc0 intel_iommu=on earlyprintk=ttyS0,115200 ' -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -bios /usr/share/qemu/OVMF.fd - TDX guest boot up successfully.
Expected Result: TDX guest can boot up successfully with debug=off and "earlyprintk".
Case ID: 019 tdx_hdevents_in5sec
Test Purpose: Collect hardware events via perf during 5 seconds and check "Performance counter stats".
Test Steps:
-
Boot host and check TDX flag.
# dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. -
Create TDX guest with pmu=on
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host,pmu=on -smp 8 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd -
After TDX guest boots up successfully, run the following command line to collect some hardware events during last 5 second.
perf stat -e cache-misses -e instructions -e branch-misses -e bus-cycles -e branch-instructions -e L1-dcache-loads -a -- sleep 5
-
Check the hardware events data, if some values are too large (say, > 1000 billions), there may be a bug.
Performance counter stats for 'system wide':
2,299 cache-misses 11,276,844,014 instructions 78,534 branch-misses # 0.00% of all branches 52,993,975,500 bus-cycles 2,818,253,634 branch-instructions 4,227,460,587 L1-dcache-loads
5.336306380 seconds time elapsed
Expected Result: Hardware events are generated correctly while TDX guest is running.
Case ID: 020 tdx_VMP_cpu_onoff
Test Purpose: To check TDX guest works when some cpu is on/off.
Test Steps:
- Boot host and check TDX flag.
# dmesg | grep -i tdx
[ 0.240761] tdx: using preloaded SEAM module
[ 3.169957] tdx: TDX initialized. - qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
- Offline some cpu on host echo 0 > /sys/devices/system/cpu/cpu17/online echo 0 > /sys/devices/system/cpu/cpu60/online echo 0 > /sys/devices/system/cpu/cpu109/online
- Check TD guest, ping TD guest
- Restore the offline cpu echo 1 > /sys/devices/system/cpu/cpu17/online echo 1 > /sys/devices/system/cpu/cpu60/online echo 1 > /sys/devices/system/cpu/cpu109/online
Expected Result: TDX works correctly with host cpu off/on
Case ID: 021 tdx_31guests
Test Purpose: Check whether 31 TD guests can bootup together.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create 1 TDX guest
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Repeat step 2, create 31 td guest in total
Expected Result: 31 TD guests can bootup together. No host kernel oops.
Case ID: 022 TDX_16G_4guests_20times
Test Purpose: Check whether 4 TD guests with 16G memory, repeat 20 times.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create 4 TDX guests with 16G memory.
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - When TDX guests boots up successfully, kill all the TD guests.
- Repeat 20 times.
Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.
Case ID: 023 tdx_uefi
Test Purpose: Check whether TDX SEAM modules are loaded by UEFI
Test Steps:
- Boot host and check TDX module in dmesg.
dmesg | grep -E "tdx: TDX module"
[ 270.134879] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20230420, build_num 507
Expected Result: TDX SEAM modules are loaded by UEFI successfully.
Case ID: 024 tdx_cpuoff_pinedVMdown
Test Purpose: Boot TD guest and pin the qemu process to a host cpu, offline the host cpu, check if host can work well.
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Create TD guest with 64 vcpu
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 64 -m 4096M -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - Pin Qemu to host cpu
pid=ps -edf | grep qemu | grep -v grep | awk '{print $2}'
taskset -pc 18 $pid - Make cpu 18 offline echo 0 > /sys/devices/system/cpu/cpu18/online
- Execute "init 0 &" in TD guest
- Check host dmesg
dmesg | tail -200 | grep hkid
dmesg | tail -200 | grep "0x8000070100000000"
No such info found in dmesg - Restore cpu 18 echo 1 > /sys/devices/system/cpu/cpu18/online
- Repeat step 2~7 20 times
Expected Result: Host works well, no crash information found
Case ID: 025 tdx_tsc_deadline_enable
Test Purpose: Disable TSC deadline on TD guests
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Boot a TD guest
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4 -m 4096M -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - TD VM can boot up successfully and tsc deadline is enabled by default
Verify that TD guest can see tsc-deadline feature:
lscpu |grep tsc_deadline_timer
Expected Result: TSC deadline can be enabled by default on TD guests.
Case ID: 026 tdx_tsc_deadline_disable
Test Purpose: Disable TSC deadline on TD guests
Test Steps:
- Boot host and check TDX flag.
## dmesg | grep -i tdx
[ 3.169957] tdx: TDX initialized. - Boot a TD guest with “-tsc-deadline”, e.g., “-cpu host,-tsc-deadline”
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host,-tsc-deadline -smp 4 -m 4096M -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd - TD VM can boot up successfully with -tsc-deadline.
Verify that TD guest cannot see tsc-deadline feature:
“lscpu |grep tsc_deadline_timer | wc -l” should output 0
Expected Result: TSC deadline can be disabled successfully on TD guests.
Case ID: 027 tdx_vsock_vm_server
Test Purpose: To check iperf works with vsock: guest as server and host as client
Test Steps:
-
On host, install iperf with vsock support.
(1) git clone https://github.com/stefano-garzarella/iperf-vsock
(2) build iperf
→ cd iperf-vsock
→ mkdir build
→ cd build
→ ../configure (If configure fails, try running./bootstrap.shfirst)
→ make
→ cd src -
Boot TD guest
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd -device vhost-vsock-pci,guest-cid=3 -
Login guest, install iperf with vsock support on guest
-
Run iperf as vsock server on guest
systemctl stop firewalld.service; systemctl disable firewalld.service
setenforce 0./iperf3 --vsock -s -
Run iperf as vsock client on host
./iperf3 --vsock -c 3 -
Get the iperf data.
Expected Result: Vsock works correctly on host and guest.
Case ID: 028 tdx_vsock_vm_client
Test Purpose: To check iperf works with vsock: guest as client and host as server
Test Steps:
-
On host, install iperf with vsock support.
(1) git clone https://github.com/stefano-garzarella/iperf-vsock
(2) build iperf
→ cd iperf-vsock
→ mkdir build
→ cd build
→ ../configure (If configure fails, try running./bootstrap.shfirst)
→ make
→ cd src -
Boot TD guest
qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::10022-:22 -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd -device vhost-vsock-pci,guest-cid=3 -
Login guest, install iperf with vsock support on guest
-
Run iperf as vsock server on host systemctl stop firewalld.service; systemctl disable firewalld.service
setenforce 0./iperf3 --vsock -s -
Run iperf as vsock client on guest
setenforce 0./iperf3 --vsock -c 2 -
Get the iperf data.
Expected Result: Vsock works correctly on host and guest.