Skip to content

Commit

Permalink
Use nextest for FPGA workflow.
Browse files Browse the repository at this point in the history
This has two benefits:

 * nextest will archive all the binaries for us, and we don't have to
   manually decide which ones to call in the workflow (a new test binary
   will be automatically invoked).

 * nextest generates the junit.xml file which can be used to inform a
   future test dashboard.
  • Loading branch information
korran committed Nov 17, 2023
1 parent 6085559 commit c08a136
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 15 deletions.
11 changes: 11 additions & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Licensed under the Apache-2.0 license

[profile.nightly]
failure-output = "immediate-final"
fail-fast = false

[profile.nightly.junit]
path = "/tmp/junit.xml"
store-success-output = true
store-failure-output = true

60 changes: 45 additions & 15 deletions .github/workflows/fpga.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,14 +133,16 @@ jobs:
- name: Build test binaries
run: |
cargo \
--config "target.aarch64-unknown-linux-gnu.rustflags = [\"-C\", \"link-arg=--sysroot=/tmp/caliptra-fpga-sysroot\"]" \
--config "target.aarch64-unknown-linux-gnu.linker = \"aarch64-linux-gnu-gcc\"" \
test --features=fpga_realtime --release --no-run --target=aarch64-unknown-linux-gnu --message-format=json > /tmp/caliptra-cargo.json
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="aarch64-linux-gnu-gcc"
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-arg=--sysroot=$FARGO_SYSROOT"
cargo nextest archive \
--features=fpga_realtime \
--release \
--target=aarch64-unknown-linux-gnu \
--archive-file=/tmp/caliptra-test-binaries.tar.zst
mkdir /tmp/caliptra-test-binaries/
cat /tmp/caliptra-cargo.json | jq -r '.executable | select(. != null)' | while read line; do
cp "$line" /tmp/caliptra-test-binaries/
done
tar xvf /tmp/caliptra-test-binaries.tar.zst -C /tmp/caliptra-test-binaries/
mksquashfs /tmp/caliptra-test-binaries /tmp/caliptra-test-binaries.sqsh -comp zstd
- name: 'Upload test binaries artifact'
Expand Down Expand Up @@ -292,6 +294,13 @@ jobs:
(needs.build_kernel_modules.result == 'success' || needs.build_kernel_modules.result == 'skipped')
steps:
- name: Checkout repo
uses: actions/checkout@v3

- name: Pull dpe submodule
run: |
git submodule update --init dpe
- name: 'Download FPGA Bitstream Artifact'
uses: actions/download-artifact@v3
with:
Expand Down Expand Up @@ -363,11 +372,32 @@ jobs:
TEST_BIN=/tmp/caliptra-test-binaries
VARS="CPTRA_UIO_NUM=4 CALIPTRA_PREBUILT_FW_DIR=/tmp/caliptra-test-firmware CALIPTRA_IMAGE_NO_GIT_REVISION=1"
sudo ${VARS} "${TEST_BIN}/caliptra_integration_tests-"*
sudo ${VARS} "${TEST_BIN}/caliptra_hw_model-"*
sudo ${VARS} "${TEST_BIN}/model_tests-"*
sudo ${VARS} "${TEST_BIN}/drivers_integration_tests-"*
sudo ${VARS} "${TEST_BIN}/rom_integration_tests-"*
sudo ${VARS} "${TEST_BIN}/fmc_integration_tests-"*
sudo ${VARS} "${TEST_BIN}/runtime_integration_tests-"*
sudo ${VARS} "${TEST_BIN}/fips_test_suite-"*
COMMON_ARGS=(
--cargo-metadata="${TEST_BIN}/target/nextest/cargo-metadata.json"
--binaries-metadata="${TEST_BIN}/target/nextest/binaries-metadata.json"
--target-dir-remap="${TEST_BIN}/target"
--workspace-remap=.
-E 'not (package(/caliptra-emu-.*/) |
package(caliptra-builder) |
package(caliptra-cfi-derive) |
package(caliptra-file-header-fix) |
package(compliance-test))'
)
cargo-nextest nextest list \
"${COMMON_ARGS[@]}" \
--message-format json > /tmp/nextest-list.json
sudo ${VARS} cargo-nextest nextest run \
"${COMMON_ARGS[@]}" \
--test-threads=1 \
--no-fail-fast \
--profile=nightly
- name: 'Upload test results'
uses: actions/upload-artifact@v3
with:
name: caliptra-test-results
path: |
/tmp/junit.xml
/tmp/nextest-list.json

0 comments on commit c08a136

Please sign in to comment.