DTS E2E tests fail on choosing firmware version

[PLangowski]

Device

QEMU

RTE version

N/A

OSFV version

develop

Affected component(s) or functionality

No response

Brief summary

The DTS E2E tests fail to choose the firmware version when deploying Dasharo.

How reproducible

No response

How to reproduce

Run any test from the E2E suite that deploys Dasharo.

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
          -v dpp_password:$DPP_PASSWORD -v dpp_download_key:$MSI_DOWNLOAD \
          -v dpp_logs_key:$MSI_LOGS -v boot_dts_from_ipxe_shell:True \
          -v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
          -t "E2E003.005*" dts/dts-e2e.robot

Expected behavior

PASS

Actual behavior

E2E003.005 MSI PRO Z690-A DDR-4 initial deployment (legacy -> Core... | FAIL |
No match found for 'Does it match your actual specification? (Y|n)' in 30 seconds. Output:

Please, select Dasharo firmware version to install:

c) Community version
d
d) DPP version (coreboot + UEFI)
b) Back to main menu

Enter an option:
Subscription version (cooreboot + EDK2) selected

Link to screenshots or logs

The logs only are available internally, as they contain DPP secrets.

Additional context

No response

Solutions you've tried

No response

[m-iwanicki]

@PLangowski You should add more information to your issues like on what branch/commit it happened. And if there was anything special that might've resulted in those tests failing you should mention it too. It would help with reproducing this error.

For example this test failed in meta-dts CI but new run resulted in different failure: https://github.com/Dasharo/meta-dts/actions/runs/12244842409/job/34159330927

E2E003.001 MSI PRO Z690-A DDR4 initial deployment (legacy -> Coreb... | PASS |
E2E003.002 MSI PRO Z690-A initial deployment (legacy -> Coreboot +... | PASS |
E2E003.003 MSI PRO Z690-A DDR-4 initial deployment (legacy -> Core... | PASS |
E2E003.004 MSI PRO Z690-A initial deployment (legacy -> Coreboot +... | PASS |
E2E003.005 MSI PRO Z690-A DDR-4 initial deployment (legacy -> Core... | PASS |
E2E003.006 MSI PRO Z690-A initial deployment (legacy -> Coreboot +... | PASS |
E2E003.007 MSI PRO Z690-A DDR-4 update (Coreboot + UEFI -> Coreboo... | PASS |
E2E003.008 MSI PRO Z690-A update (Coreboot + UEFI -> Coreboot + UE... | PASS |
E2E003.009 MSI PRO Z690-A DDR-4 update (Coreboot + UEFI -> Coreboo... | PASS |
E2E003.010 MSI PRO Z690-A update (Coreboot + UEFI -> Coreboot + UE... | PASS |
Dts-E2E                                                               | PASS |
E2E003.011 MSI PRO Z690-A DDR4 transition (Coreboot + UEFI -> head... | PASS |
E2E003.012 MSI PRO Z690-A DDR4 transition (Coreboot + UEFI -> head... | PASS |
E2E003.013 MSI PRO Z690-A transition (UEFI -> heads) - without cre... | PASS |
E2E003.014 MSI PRO Z690-A transition (UEFI -> heads) - with creden... | PASS |
Dts-E2E                                                               | PASS |
E2E004.001 Dell OptiPlex 7010 DPP initial deployment (legacy -> Co... | PASS |
E2E004.002 Dell Optiplex 7010 DPP initial deployment (legacy -> Co... | PASS |
E2E004.003 Dell Optiplex 7010 DPP update (Coreboot + UEFI -> Coreb... | PASS |
E2E004.004 Dell Optiplex 7010 DPP update (Coreboot + UEFI -> Coreb... | PASS |
E2E004.005 Dell OptiPlex 9010 DPP initial deployment (legacy -> Co... | PASS |
E2E004.006 Dell Optiplex 9010 DPP initial deployment (legacy -> Co... | PASS |
E2E004.007 Dell Optiplex 9010 DPP update (Coreboot + UEFI -> Coreb... | PASS |
E2E004.008 Dell Optiplex 9010 DPP update (Coreboot + UEFI -> Coreb... | PASS |
Dts-E2E                                                               | PASS |
E2E002.001 NCM NV4XMB,ME,MZ initial deployment (legacy -> Coreboot... | PASS |
E2E002.002 NCM NS50_70MU initial deployment (legacy -> Coreboot + ... | PASS |
E2E002.003 NCM NS5x_NS7xPU initial deployment (legacy -> Coreboot ... | PASS |
E2E002.004 NCM NV4xPZ initial deployment (legacy -> Coreboot + UEF... | PASS |
E2E002.005 NCM NV4xPZ transition (Coreboot + UEFI -> Coreboot + He... | PASS |
E2E002.006 NCM transition NV4xPZ (Coreboot + UEFI -> Heads) - DPP ... | FAIL |
E2E002.007 NCM V540_6x_TU initial deployment (legacy -> Coreboot +... | PASS |
E2E002.008 NCM V560_6x_TU initial deployment (legacy -> Coreboot +... | PASS |
E2E002.009 NCM V540TNC_TND_TNE initial deployment (legacy -> Coreb... | PASS |
E2E002.010 NCM V560TNC_TND_TNE initial deployment (legacy -> Coreb... | PASS |
Dts-E2E                                                               | FAIL |

And from what I can see there were no changes to workflow or tests. So failure might've happened due to timing issue (e.g. waiting some arbitrary amount of time instead of waiting for expected output).
Full error with failing test:

------------------------------------------------------------------------------
E2E002.006 NCM transition NV4xPZ (Coreboot + UEFI -> Heads) - DPP ... | FAIL |
No match found for 'If you did not buy any DPP
packages - feel free to continue.
Press Enter to continue' in 30 seconds. Output:
Enter logs key:                ********

Enter firmware download key:

Enter password:                *****
Dasharo DPP credentials have been saved
Checking for Dasharo firmware...
Waiting for network connection ...
Network connection have been established!
Checking if board is Dasharo compatible.
Waiting for network connection ...
Network connection have been established!
/usr/sbin/dts-subscription.sh: line 60: [: -eq: unary operator expected
Something may be wrong with the DPP credentials or you may not
have access to Dasharo Firmware. If so, consider getting Dasharo
Subscription and improving security of your platform!
Press Enter to continue.
------------------------------------------------------------------------------

[EduKav1813]

@PLangowski I try to but I cannot reproduce this issue on: dts 2.1.2 (commit 86fd10102a3b0c0c9adb652dbbd67efbf7ddc5c0) + patch:

diff --git a/meta-dts-distro/recipes-dts/dts-scripts/dts-scripts_git.bb b/meta-dts-distro/recipes-dts/dts-scripts/dts-scripts_git.bb
index bd9efa05be39..df0b1c35b104 100644
--- a/meta-dts-distro/recipes-dts/dts-scripts/dts-scripts_git.bb
+++ b/meta-dts-distro/recipes-dts/dts-scripts/dts-scripts_git.bb
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://LICENSES/Apache-2.0.txt;md5=c846ebb396f8b174b10ded477
 
 PV = "0.1+git${SRCPV}"
 
-SRC_URI = "git://github.com/Dasharo/dts-scripts;protocol=https;branch=main"
+SRC_URI = "git://github.com/Dasharo/dts-scripts;protocol=https;branch=fix-check-for-dasharo-firmware"
 SRCREV = "27df8825c68d6a356188e51e8bdff0428aa0ae89"
 
 S = "${WORKDIR}/git"

[DaniilKl]

@EduKav1813, any results from tests you are basing your answer on here? Check out how Michał presented his results.

[m-iwanicki]

@EduKav1813 Let's wait for test results of 2.1.2 release as from what I saw errors happen in different places now if at all.

[m-iwanicki]

@EduKav1813 https://github.com/Dasharo/meta-dts/actions/runs/12580633445 every test succeeded