Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New build system #4

Merged
merged 32 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
d76c264
propose new build system; 2 examples
mkannwischer Jul 11, 2024
70cb440
update slothy
mkannwischer Jul 11, 2024
441e131
add CI build
mkannwischer Jul 12, 2024
d22a52d
fix nproc
mkannwischer Jul 12, 2024
8465efb
git ignore build dirs
mkannwischer Jul 12, 2024
4831124
remove obsolete init scripts
mkannwischer Jul 12, 2024
61e0a76
add CI running tests on qemu platforms
mkannwischer Jul 12, 2024
e2bce9e
break test
mkannwischer Jul 12, 2024
59df819
build one elf for each test/platform combination
mkannwischer Jul 12, 2024
56ec599
CI: use more threads for make
mkannwischer Jul 12, 2024
70a9aa2
add support to build standalone artifacts
mkannwischer Jul 12, 2024
6ee5cbf
don't need qemu here
mkannwischer Jul 12, 2024
85f29a4
refactor names
mkannwischer Jul 12, 2024
06d488f
tests checking for last line of output
mkannwischer Jul 15, 2024
fdb0b89
change back dilithium test
mkannwischer Jul 15, 2024
2efc615
move common test files back to the tests folder
mkannwischer Jul 15, 2024
d2c37d5
change targets to <PLATFORM>_<TEST> + disallow _ in test name;
mkannwischer Jul 15, 2024
725e7cf
Revert "add support to build standalone artifacts"
mkannwischer Jul 18, 2024
0cf72f4
add other tests
mkannwischer Jul 18, 2024
fb467be
add other tests
mkannwischer Jul 18, 2024
7d4ee10
disable build that overflows flash
mkannwischer Jul 18, 2024
fc76c5c
move {poly,misc}.{c,h} back to common
mkannwischer Jul 18, 2024
2cf68e8
remove more duplicated assembly
mkannwischer Jul 18, 2024
587f49c
remove more duplicate assembly files
mkannwischer Jul 18, 2024
e208dfe
add missing unpack test
mkannwischer Jul 18, 2024
90e486d
separate build artifact by target elf to work around collisions
mkannwischer Jul 18, 2024
d3d51b1
remove more duplicate assembly
mkannwischer Jul 18, 2024
b1ec15b
fix m85
mkannwischer Jul 19, 2024
e014bdc
M85: switch over to SRAM, so we do not run into RAM limitations
mkannwischer Jul 19, 2024
f06b3a0
remove currently unsupported platforms
mkannwischer Jul 19, 2024
8aac3d5
disable intmulntt test for m85-an555
mkannwischer Jul 22, 2024
9c2295f
update README
mkannwischer Jul 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
16 changes: 16 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Build everything
on:
pull_request:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: install dependencies
run: sudo apt install gcc-arm-none-eabi
- name: Make all
run: |
make -j$(nproc)
16 changes: 16 additions & 0 deletions .github/workflows/check.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Run tests on qemu platforms
on:
pull_request:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: install dependencies
run: sudo apt install gcc-arm-none-eabi qemu-system
- name: Make all
run: |
make -j$(nproc) check
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
**/build/
**/*.elf
**/*.bin
1,981 changes: 65 additions & 1,916 deletions Makefile

Large diffs are not rendered by default.

57 changes: 19 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ of post-quantum cryptography targeting Cortex-M4, with a focus on CPUs implement
Helium&trade; Technology), such as the [Arm<sup>&reg;</sup>
Cortex&trade;-M55](https://www.arm.com/products/silicon-ip-cpu/cortex-m/cortex-m55) processor.

### SLOTHY + HeLight
### SLOTHY

This repository also contains the source code for the SLOTHY/HeLight assembly superoptimizer, discussed in the paper [Fast and Clean: Auditable high-performance assembly via constraint solving](https://eprint.iacr.org/2022/1303). See [helight/README.md](helight/README.md) for more information.
This repository also contains the source code for the SLOTHY assembly superoptimizer, discussed in the paper [Fast and Clean: Auditable high-performance assembly via constraint solving](https://eprint.iacr.org/2022/1303). See [slothy/README.md](slothy/README.md) for more information.

### M-Profile Vector Extension (MVE)

Expand All @@ -36,7 +36,7 @@ The main components of the repository are the following:
* [`asm`](asm): Core primitives in optimized assembly, mostly auto-generated.
* [`tests`](tests): C-based tests for core primitives using a minimal hardware abstraction layer (HAL).
* [`envs`](envs): Test environments implementing the HAL.
* [`helight`](helight): The SLOTHY/HeLight assembly superoptimizer. See the [README](helight/README.md) for more information.
* [`slothy`](slothy): The SLOTHY assembly superoptimizer. See the [README](slothy/README.md) for more information.

The following sections explain each component in greater detail.

Expand Down Expand Up @@ -78,17 +78,21 @@ As mentioned above, the tests from [`tests/`](tests/) can be run in any environm
interface [`tests/inc/hal.h`](tests/inc/hal.h). This flexibility is useful in order to test the MVE assembly in different models or
simulators of MVE-implementations.

The supported test environments are located in [`envs`](envs/). For now, there are two test environments for testing
functional behaviour, based on the freely available FVPs for the Arm<sup>&reg;</sup> Corstone&trade;-300 MPS2 and Arm<sup>&reg;</sup> Corstone&trade;-300 MPS3. See
[`envs/fvp-corstone300-mps2/`](./envs/fvp-corstone300-mps2/) and [`envs/fvp-corstone300-mps3/`](./envs/fvp-corstone300-mps3/) for more information, including build and usage instructions.
The supported test environments are located in [`envs`](envs/).
As of now, we are supporting two platforms:
- [Arm® Corstone™ SSE-300 with Cortex®-M55 and Ethos™-U55 (AN547)](https://developer.arm.com/downloads/view/AN547)
- [Arm® Corstone™ SSE-310 with Cortex®-M85 and Ethos™-U55 (AN555)](https://developer.arm.com/downloads/view/AN555)

The former can be emulated using qemu (>=6.0).
Previously, the freely available FVPs for the Arm<sup>&reg;</sup> Corstone&trade;-300 MPS2 and Arm<sup>&reg;</sup> Corstone&trade;-300 MPS3 were also supported.
However, these are currently no longer maintained (see https://github.com/slothy-optimizer/pqmx/issues/7).

Writing a new test environment requires the provisioning of build, run and debug scripts, plus an implementation of the
test HAL [`tests/inc/hal.h`](tests/inc/hal.h). The actual test sources can then be moved into the test environment through a symbolic
link. This way, there's no need to each test for each test environment, but instead a choice of test environment + test
can be run by linking the test into the environment's template and building/running it. If you have added a new test
test HAL [`tests/inc/hal.h`](tests/inc/hal.h).
If you have added a new test
environment, you can test that it works against the HelloWorld test in [`tests/helloworld`](tests/helloworld/).

To run the tests in qemu, the target `run-m55-an547-{test_name}` can be used. It will build the executable from the sources and run it using `qemu-system-arm -M mps3-an547 -nographic -semihosting -kernel`.
To run the tests in qemu, the target `run-m55-an547_{test_name}` can be used. It will build the executable from the sources and run it using `qemu-system-arm -M mps3-an547 -nographic -semihosting -kernel`.

## License

Expand All @@ -98,46 +102,23 @@ The software is provided under an MIT license. Contributions to this project are

### Prerequisites

#### Code generation

Code generation requires Python 3. Call `init.sh` to build a Python environment suitable for running the Python scripts
in this repository, and run `source envsetup.sh` to enter it and set the encessary environment variables.

#### Compilation

Compilation requires a toolchain supporting Armv8.1-M, such as Arm Compiler 6.14 or GNU Arm Embedded Toolchain 10-2020-q4-major or
higher. See [README](./envs/fvp-corstone300-mps2/README.md) for installation instructions.

#### Corstone-300 MPS2/MPS3 FVP test environment

In addition to an Armv8.1-M capable compiler, the Corstone-300 MPS2/MPS3 FVP test environment requires CMSIS 5 plus the corresponding CMSIS pack, and
the FVP itself. See the corresponding README ([MPS2](./envs/fvp-corstone300-mps3/README.md),[MPS2](./envs/fvp-corstone300-mps3/README.md)) for detailed instructions.
higher.

### Usage flow

Make sure you have entered the Python environment via `source envsetup.sh`.

The code in this repository can then be generated, compiled and run via `make`:

* `make codegen` generates all auto-generated assembly in `asm/auto` and copies them to the relevant tests `tests/*/auto/`.
* `make {build,run,debug}-{m55-mps2-fvp,m55-mps3-fvp}-{helloworld,ntt,schoolbook,transpose,montgomery,permute,toom}` builds/runs/debugs the chosen
test in the chosen test environment. For example, `make
run-m55-mps3-fvp-schoolbook` will generate the MVE assembly for schoolbook multiplication and build and run it in the
Cortex-M55 FVP test environment.
* `make {build,run}-{m55-an547,m85-an555}-{helloworld,ntt-kyber,ntt-dilithium}` builds/runs the chosen
test in the chosen test environment.

We recommend trying

```
make run-m55-mps3-fvp-helloworld
make run-m55-an547_helloworld
```

after setting up the required tooling for the Corstone-300 FVP test environment, to check that the tools are in the
after setting up the required tooling, to check that the tools are in the
right place and working as expected.

#### Choice of compiler

By default, Arm Compiler will be used. If you wish to use GCC instead, supply `USE_GCC=1` to the above invocations of
`make`, e.g. `make USE_GCC=1 run-m55-mps3-fvp-helloworld`.

Note that if you have previously built any tests with a different compiler, you will need to run `make clean` to remove
incorrect build artifacts.
1 change: 1 addition & 0 deletions asm/manual/fx_fft/base_symbolic.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M55_unroll1.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M55_unroll2.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M55_unroll4.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M85_unroll1.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M85_unroll2.s
2 changes: 1 addition & 1 deletion asm/manual/sqmag/cmplx_mag_sqr_fx_opt_M85_unroll4.s
File renamed without changes.
Binary file removed envs/core/Cortex-M55.axf
Binary file not shown.
89 changes: 0 additions & 89 deletions envs/core/Makefile

This file was deleted.

Empty file removed envs/core/build/test_common/dummy
Empty file.
Empty file.
Empty file.
Empty file.
6 changes: 0 additions & 6 deletions envs/core/inc/hal_env.h

This file was deleted.

1 change: 0 additions & 1 deletion envs/core/inc/test_inc

This file was deleted.

Loading
Loading